-- Logs begin at Thu 2022-07-28 16:48:23 UTC, end at Thu 2022-07-28 18:07:05 UTC. -- Jul 28 17:29:54.473664 dv-cinder-813229-13-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 28 17:29:54.481637 dv-cinder-813229-13-iscsi devstack@keystone.service[80315]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 28 17:29:54.483745 dv-cinder-813229-13-iscsi devstack@keystone.service[80315]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Jul 28 17:29:54 2022] *** Jul 28 17:29:54.483745 dv-cinder-813229-13-iscsi devstack@keystone.service[80315]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 28 17:29:54.483745 dv-cinder-813229-13-iscsi devstack@keystone.service[80315]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 28 17:29:54.483745 dv-cinder-813229-13-iscsi devstack@keystone.service[80315]: nodename: dv-cinder-813229-13-iscsi Jul 28 17:29:54.483745 dv-cinder-813229-13-iscsi devstack@keystone.service[80315]: machine: x86_64 Jul 28 17:29:54.483745 dv-cinder-813229-13-iscsi devstack@keystone.service[80315]: clock source: unix Jul 28 17:29:54.483745 dv-cinder-813229-13-iscsi devstack@keystone.service[80315]: pcre jit disabled Jul 28 17:29:54.483745 dv-cinder-813229-13-iscsi devstack@keystone.service[80315]: detected number of CPU cores: 4 Jul 28 17:29:54.483745 dv-cinder-813229-13-iscsi devstack@keystone.service[80315]: current working directory: / Jul 28 17:29:54.483745 dv-cinder-813229-13-iscsi devstack@keystone.service[80315]: detected binary path: /usr/bin/uwsgi-core Jul 28 17:29:54.484302 dv-cinder-813229-13-iscsi devstack@keystone.service[80315]: your processes number limit is 47680 Jul 28 17:29:54.484302 dv-cinder-813229-13-iscsi devstack@keystone.service[80315]: your memory page size is 4096 bytes Jul 28 17:29:54.484302 dv-cinder-813229-13-iscsi devstack@keystone.service[80315]: detected max file descriptor number: 2048 Jul 28 17:29:54.484302 dv-cinder-813229-13-iscsi devstack@keystone.service[80315]: lock engine: pthread robust mutexes Jul 28 17:29:54.484302 dv-cinder-813229-13-iscsi devstack@keystone.service[80315]: thunder lock: enabled Jul 28 17:29:54.484302 dv-cinder-813229-13-iscsi devstack@keystone.service[80315]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 28 17:29:54.484302 dv-cinder-813229-13-iscsi devstack@keystone.service[80315]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Jul 28 17:29:54.526528 dv-cinder-813229-13-iscsi devstack@keystone.service[80315]: Python main interpreter initialized at 0x55a434cb19c0 Jul 28 17:29:54.526528 dv-cinder-813229-13-iscsi devstack@keystone.service[80315]: python threads support enabled Jul 28 17:29:54.526528 dv-cinder-813229-13-iscsi devstack@keystone.service[80315]: your server socket listen backlog is limited to 100 connections Jul 28 17:29:54.526528 dv-cinder-813229-13-iscsi devstack@keystone.service[80315]: your mercy for graceful operations on workers is 90 seconds Jul 28 17:29:54.527115 dv-cinder-813229-13-iscsi devstack@keystone.service[80315]: mapped 403077 bytes (393 KB) for 2 cores Jul 28 17:29:54.527115 dv-cinder-813229-13-iscsi devstack@keystone.service[80315]: *** Operational MODE: preforking *** Jul 28 17:29:54.527320 dv-cinder-813229-13-iscsi devstack@keystone.service[80315]: *** uWSGI is running in multiple interpreter mode *** Jul 28 17:29:54.527320 dv-cinder-813229-13-iscsi devstack@keystone.service[80315]: spawned uWSGI master process (pid: 80315) Jul 28 17:29:54.527546 dv-cinder-813229-13-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 28 17:29:54.528392 dv-cinder-813229-13-iscsi devstack@keystone.service[80315]: spawned uWSGI worker 1 (pid: 80324, cores: 1) Jul 28 17:29:54.528495 dv-cinder-813229-13-iscsi devstack@keystone.service[80315]: spawned uWSGI worker 2 (pid: 80325, cores: 1) Jul 28 17:29:54.529536 dv-cinder-813229-13-iscsi devstack@keystone.service[80315]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 28 17:30:00.076781 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 28 17:30:00.076781 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: from cryptography.utils import int_from_bytes Jul 28 17:30:00.077638 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 28 17:30:00.077638 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: from cryptography.utils import int_from_bytes Jul 28 17:30:00.089756 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 28 17:30:00.089756 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: from cryptography.utils import int_from_bytes Jul 28 17:30:00.090247 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 28 17:30:00.090247 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: from cryptography.utils import int_from_bytes Jul 28 17:30:00.128423 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 28 17:30:00.128607 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 28 17:30:00.128789 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] command line args: [] {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 28 17:30:00.129064 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 28 17:30:00.129267 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] ================================================================================ {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 28 17:30:00.129527 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] admin_token = **** {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:30:00.129776 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] config_dir = [] {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:30:00.129966 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:30:00.130149 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] config_source = [] {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:30:00.130331 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] debug = True {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:30:00.130531 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:30:00.131900 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:30:00.131900 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:30:00.131900 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:30:00.131900 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:30:00.131900 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] list_limit = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:30:00.132222 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] log_config_append = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:30:00.132222 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:30:00.132331 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] log_dir = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:30:00.132480 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] log_file = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:30:00.132687 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:30:00.132942 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:30:00.133223 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:30:00.133480 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:30:00.133660 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:30:00.133830 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:30:00.134076 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:30:00.134326 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:30:00.134575 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:30:00.134800 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:30:00.135035 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:30:00.135277 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:30:00.135528 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:30:00.135778 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:30:00.136058 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:30:00.136563 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.89/identity {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:30:00.136747 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] publish_errors = False {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:30:00.136982 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:30:00.137321 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:30:00.137507 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:30:00.137664 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 28 17:30:00.137764 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:30:00.137836 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 28 17:30:00.137992 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:30:00.138087 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] command line args: [] {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 28 17:30:00.138233 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] standard_threads = False {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:30:00.138326 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 28 17:30:00.138442 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] ================================================================================ {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 28 17:30:00.138534 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:30:00.138697 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] admin_token = **** {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:30:00.138798 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:30:00.138884 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:30:00.138962 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] config_dir = [] {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:30:00.139040 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] use_journal = False {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:30:00.139113 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:30:00.139190 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] use_json = False {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:30:00.139257 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] config_source = [] {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:30:00.139405 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] use_stderr = False {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:30:00.139512 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] debug = True {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:30:00.139595 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] use_syslog = False {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:30:00.139676 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:30:00.139772 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:30:00.139845 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:30:00.140012 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:30:00.140088 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.140252 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.140339 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:30:00.140498 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:30:00.140581 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.141283 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] list_limit = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:30:00.141560 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.141987 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] log_config_append = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:30:00.142068 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.142139 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:30:00.142139 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] log_dir = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:30:00.142263 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.142339 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] log_file = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:30:00.142417 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.142488 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:30:00.142563 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.142633 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:30:00.142719 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.142799 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:30:00.142917 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.143001 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:30:00.143109 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:30:00.143186 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.143284 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:30:00.143412 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.143528 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:30:00.143645 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.143772 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:30:00.143874 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.144003 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:30:00.144128 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.144232 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:30:00.144380 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.144461 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:30:00.144623 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.144708 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:30:00.144884 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.144962 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:30:00.145136 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:30:00.145281 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.145398 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.145523 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:30:00.145644 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.145886 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.146008 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.89/identity {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:30:00.146122 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] publish_errors = False {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:30:00.146245 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.146359 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:30:00.146471 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.146589 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:30:00.146701 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.146822 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:30:00.146936 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:30:00.147079 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.147218 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:30:00.147332 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.147443 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] standard_threads = False {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:30:00.147566 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.147686 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.147798 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:30:00.147947 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.148172 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:30:00.148288 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.148434 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.148549 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:30:00.148671 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.148785 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] use_journal = False {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:30:00.148895 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.149117 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] use_json = False {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:30:00.149363 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.149487 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.149605 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] use_stderr = False {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:30:00.149720 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.149832 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] use_syslog = False {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:30:00.149949 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.150102 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.150266 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 17:30:00.150382 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.150565 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.150717 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.150834 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.151025 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.151179 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.151295 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.151488 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.151677 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.151802 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.151964 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.152137 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.152255 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.152428 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.152585 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.152709 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.152898 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.153013 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.153150 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.153408 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.153523 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.153709 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.153832 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.153947 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.154058 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.154168 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.154279 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.154398 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.154512 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.154623 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.154740 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.154852 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.154964 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.155087 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.155200 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.155317 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.155436 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.155552 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.155670 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.155884 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.156003 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.156133 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.156254 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.156394 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.157542 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.157679 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.157796 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.157909 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.158022 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.158141 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.158255 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.158376 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.158491 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.158603 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.158715 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.158811 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.158904 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.159007 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.159110 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.159205 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.159308 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.159408 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.159503 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.159503 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.159694 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.159789 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.159884 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.159993 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.160142 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.160258 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.160371 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.160483 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.160584 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.160694 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.160815 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.160937 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.161051 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.161179 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.161295 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.161417 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.161531 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.161653 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.161775 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.161889 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.162010 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.162123 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.162241 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.162355 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.162475 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.162589 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.162712 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.162832 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.162832 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.163034 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.163155 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.163268 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.163382 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.163494 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.163611 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.163725 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.163846 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.163959 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.164068 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.164186 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.164299 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.164402 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.164521 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.164634 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.164746 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.164857 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.164968 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.165080 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.165226 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.165349 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.165470 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.165584 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.165699 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.165812 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.165932 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.166046 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.166158 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.166270 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.166388 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.166508 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.166630 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.166791 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] auth.external = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.166908 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.167021 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.167132 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.167243 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.167365 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.167485 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.167612 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.167727 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] auth.password = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.167839 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.167952 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] auth.token = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.168138 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.168261 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.168375 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.168487 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.168607 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.168720 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.168841 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.168965 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.169086 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.169214 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.169330 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.169443 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.169555 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.169680 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.169801 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.169915 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] credential.caching = True {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.170027 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.170140 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.170252 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.170372 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.170486 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.170598 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.170725 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.170848 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.170968 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.170968 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.171190 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.171303 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.171415 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.171526 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.171638 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.171759 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.171873 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.171985 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.172262 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.172379 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.172492 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.172617 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.172743 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.172743 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.172972 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.173085 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.173254 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.173372 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.173476 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.173580 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.173694 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.173815 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] federation.caching = True {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.173930 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.174051 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.174165 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.174286 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.174400 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.174400 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.174612 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.174725 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.174839 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.174959 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.175074 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.175187 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.175300 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.175419 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.175537 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.175661 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.175776 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.175889 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.176008 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.176123 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.176242 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.176365 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.176479 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.176598 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.176712 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] identity.caching = True {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.176826 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.176939 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.177061 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.177188 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.177304 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] auth.external = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.177417 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.177529 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.177642 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.177841 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.177961 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.178076 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.178188 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.178301 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] auth.password = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.178420 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.178544 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] auth.token = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.178659 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.178780 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.178895 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.179007 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.179121 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.179242 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.179357 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.179475 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.179590 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.179703 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.179824 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.179939 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.180052 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.180166 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.180278 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.180391 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] credential.caching = True {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.180515 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.180631 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.180751 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.180875 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.180990 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.181104 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.181278 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.181396 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.181509 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.181615 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.181727 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.181840 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.182027 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.182144 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.182259 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.182372 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.182484 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.182597 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.182719 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.182834 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.182947 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.183059 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.183173 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.183286 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.183408 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.183522 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.183635 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.183748 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.183860 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.183973 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.184094 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.184200 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] federation.caching = True {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.184312 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.184425 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.184538 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.184650 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.184772 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.184886 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.185000 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.185112 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.185239 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.185358 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.185489 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.185605 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.185719 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.185832 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.185945 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.186058 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.186180 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.186302 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.186419 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.186533 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.186646 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.186759 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.186880 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.186995 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] identity.caching = True {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.187107 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.187220 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.187334 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.187448 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.187570 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.188179 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.188696 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.188814 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.188927 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.189040 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.189040 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.189321 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.189321 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.189537 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.189650 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] ldap.user = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.189764 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.189894 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.190010 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.190123 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.190237 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.190350 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.190462 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.190585 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.190700 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.190814 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.190927 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.191046 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.191169 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.191284 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.191397 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.191510 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.191624 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.191737 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.191859 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.191974 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.192087 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.192201 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.192313 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.192476 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.192600 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.192716 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.192835 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.192949 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.193061 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.193191 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.193319 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.193319 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.193532 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.193645 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.193765 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.193878 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.193999 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.194112 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.194224 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.194336 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.194448 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.194559 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.194679 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.194794 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.194906 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.195026 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.195139 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.195251 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.195371 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.195490 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.195602 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.195714 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.195827 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.195938 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.196059 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.196172 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.196284 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.196404 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.196516 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.196634 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.196754 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.196869 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.196981 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.197099 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.197264 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.197378 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.197502 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.197617 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.197729 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.197840 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.197953 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.198065 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.198185 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.198299 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.198411 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] resource.caching = True {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.198523 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.198634 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.198745 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.198866 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.198980 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.198980 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.199190 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.199302 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.199414 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.199534 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] ldap.user = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.199647 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.199759 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.199871 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.199982 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.200095 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.200216 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.200329 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.200440 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.200552 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.200663 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.200774 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] role.caching = True {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.200894 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.201008 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] role.driver = sql {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.201128 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.201258 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.201372 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.201486 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.201612 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.201728 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.201842 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.201954 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.202074 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.202189 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.202311 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.202425 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.202538 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.202651 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.202763 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.202884 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.202999 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.203112 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.203244 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.204508 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.204640 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.204766 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.204883 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.204997 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.205112 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.205272 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.205388 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.205514 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.205631 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.205631 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.205845 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.205845 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.206059 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.206252 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.206372 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.206486 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.206599 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.206712 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.206825 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.206947 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.207063 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.207176 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.207289 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.207403 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.207516 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.207516 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.207743 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.207857 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.207971 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.207971 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.208183 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.208305 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.208421 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.208534 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.208648 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.208761 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.208874 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.208996 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.209111 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.209240 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.209354 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] resource.caching = True {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.209468 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.209581 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.209707 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.209822 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.209935 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.210048 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.210161 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] token.caching = True {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.210274 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.210396 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.210511 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.210625 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.210738 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.210850 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.210963 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.211084 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.211199 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.211312 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.211424 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.211538 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.211650 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] role.caching = True {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.211775 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.211890 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] role.driver = sql {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.212004 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.212116 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.212229 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.212341 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.212467 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.212583 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.212697 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.212810 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.212922 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.213034 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.213203 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.213321 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.213435 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.213548 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.213661 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.213787 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.213903 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.214016 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.214016 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.214228 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.214341 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.214453 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] database.connection = **** {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.214575 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.214702 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.214816 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.214930 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.215043 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.215163 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.215163 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.215386 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.215501 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.215620 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.215735 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.215848 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.215961 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.216086 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.216201 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.216314 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.216427 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.216540 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.216654 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.216776 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.216891 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.217004 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.217117 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.217247 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.217361 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.217486 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.217602 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.217715 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.217829 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.217942 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.218063 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.218178 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.218291 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.218403 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.218517 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.218629 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.218750 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80325) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 28 17:30:00.218909 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.218909 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.218909 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.218909 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.218909 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] token.caching = True {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.218909 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.219713 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.219713 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.219935 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.220119 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.220364 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.220612 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.220882 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.221100 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.221362 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.221601 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.221835 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.222082 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.222331 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.222561 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.222802 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.223102 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.223364 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] database.connection = **** {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.223609 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.223856 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.224093 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.224335 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.224571 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.224806 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.225042 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.225301 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.225548 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.225783 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.226027 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.226262 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.226509 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.226743 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.226988 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.227217 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.227457 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.227690 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 17:30:00.227859 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80324) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 28 17:30:00.246885 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80325) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:30:00.247116 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80325) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:30:00.247353 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80325) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:30:00.252941 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80324) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:30:00.253191 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80324) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:30:00.253408 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80324) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:30:00.256266 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80325) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:30:00.261640 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80324) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:30:00.286959 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80325) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:30:00.287184 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80325) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:30:00.301762 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80324) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:30:00.301994 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80324) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:30:00.340934 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80325) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:30:00.341587 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80324) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:30:00.392649 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80325) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:30:00.392806 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80324) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:30:00.392901 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80325) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:30:00.392969 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80324) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:30:00.457181 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80324) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:30:00.457428 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80324) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:30:00.457650 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80324) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:30:00.457756 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80325) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:30:00.457866 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80324) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:30:00.457951 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80325) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:30:00.458082 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80324) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:30:00.458166 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80325) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:30:00.458294 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80324) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:30:00.458388 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80325) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:30:00.458587 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80325) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:30:00.458808 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80325) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:30:00.480977 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80324) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:30:00.481223 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80324) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:30:00.481339 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80325) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:30:00.481452 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80324) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:30:00.481540 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80325) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:30:00.481668 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80324) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:30:00.481753 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80325) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:30:00.481878 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80324) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:30:00.481964 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80325) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:30:00.482086 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80324) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:30:00.482170 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80325) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:30:00.482295 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80324) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:30:00.482387 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80325) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:30:00.482536 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80325) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:30:00.493308 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80324) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:30:00.493662 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80325) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:30:00.507359 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80325) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:30:00.507558 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80324) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:30:00.507772 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80325) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:30:00.507896 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80324) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:30:00.508061 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80325) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:30:00.508143 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80324) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:30:00.508304 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80324) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:30:00.508424 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80325) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:30:00.508504 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80324) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:30:00.508579 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80325) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:30:00.508710 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80324) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:30:00.508801 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80325) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:30:00.508970 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80324) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:30:00.509102 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80325) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 17:30:00.526040 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.526253 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 28 17:30:00.526591 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.526803 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 28 17:30:00.526911 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.527054 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.527138 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 28 17:30:00.527311 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.527424 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.527567 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.527651 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 28 17:30:00.527820 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.527902 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.528117 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.528371 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.528622 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.537533 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.537809 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.538078 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.538360 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.538458 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.538686 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.538952 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.539230 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.545274 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80325) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 17:30:00.546200 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80324) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 17:30:00.550580 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80325) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 17:30:00.550859 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.551141 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.551422 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.551506 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80324) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 17:30:00.551702 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.551788 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.551960 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.552044 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.552226 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.552317 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.552496 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.552579 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.552748 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.552828 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.553011 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.553094 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.553300 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.553385 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.553565 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.553828 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.554088 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.576253 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.576833 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.577775 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80325) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 17:30:00.578056 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.578939 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80324) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 17:30:00.579407 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.583428 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80325) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 17:30:00.583706 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80324) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 17:30:00.583789 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.583980 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.584064 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.584254 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.590588 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80325) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 17:30:00.590874 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.590998 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80324) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 17:30:00.591146 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.595717 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80325) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 17:30:00.595819 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80324) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 17:30:00.596000 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.596084 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.596285 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.596370 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.596568 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.596651 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.596844 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.596935 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.597106 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.597208 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.597402 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.597485 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.597674 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.597765 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.613593 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.613823 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.613908 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.614091 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.617775 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80325) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 17:30:00.617992 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80324) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 17:30:00.621185 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.621368 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.621499 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.621662 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.625977 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80325) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 17:30:00.626259 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80324) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 17:30:00.629128 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80325) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 17:30:00.629426 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80324) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 17:30:00.632301 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.632560 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.632643 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.632852 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.632933 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.633128 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.633234 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.633431 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.633513 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.633715 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.633810 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.633988 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.650310 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80324) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 17:30:00.650409 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80325) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 17:30:00.650579 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.650677 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.650837 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.650918 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.651090 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.651171 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.658791 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.659156 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.660342 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.660614 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.660711 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.660928 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.663462 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80325) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 17:30:00.663740 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.663826 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80324) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 17:30:00.664037 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.664122 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.664316 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.664409 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.664598 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.664684 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.664877 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.676294 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80325) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 17:30:00.676430 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80324) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 17:30:00.676575 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.676671 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.676853 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.676946 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.677133 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.677235 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.677434 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.677517 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.677702 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.677789 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.677976 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.678058 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.693439 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80324) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 17:30:00.693704 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80325) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 17:30:00.696381 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80324) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 17:30:00.696646 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80325) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 17:30:00.699315 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.699614 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.701915 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.702153 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.704022 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80324) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 17:30:00.704283 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80325) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 17:30:00.704420 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.704574 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.704707 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.704855 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.711218 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.711353 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.712810 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80324) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 17:30:00.712961 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80325) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 17:30:00.715715 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.715905 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.716009 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.716196 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.716280 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.716481 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.716568 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.716759 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.725367 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80324) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 17:30:00.725473 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80325) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 17:30:00.725647 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.725757 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.725919 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.726029 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.732817 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80324) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 17:30:00.732922 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80325) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 17:30:00.733084 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.733197 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.733379 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.733463 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.733648 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.733745 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.733922 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.734005 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.734195 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.734278 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.734500 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.734623 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.734766 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.734850 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.735181 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.735315 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.735467 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.735551 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.735746 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.735828 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.736005 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.736089 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.736265 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.736347 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.736521 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80324) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.736611 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80325) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 17:30:00.770629 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55a434cb19c0 pid: 80324 (default app) Jul 28 17:30:00.770874 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55a434cb19c0 pid: 80325 (default app) Jul 28 17:30:00.773953 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d3827e-f572-4566-a182-03c0eac892f3 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:00.774237 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d3827e-f572-4566-a182-03c0eac892f3 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:00.774511 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d3827e-f572-4566-a182-03c0eac892f3 None None] PATH_INFO: `/v3` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:00.778941 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1/1] 10.222.0.89 () {50 vars in 791 bytes} [Thu Jul 28 17:30:00 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:30:08.460496 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f61f1e-99e5-45b5-80c9-4f68d1e7bc4f None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:08.461066 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f61f1e-99e5-45b5-80c9-4f68d1e7bc4f None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:08.461066 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f61f1e-99e5-45b5-80c9-4f68d1e7bc4f None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:08.465583 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1/2] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:30:08 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 17:30:08.471302 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1e97c3-9f7c-4b60-b1e4-1e3569dde6ee None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:08.471709 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1e97c3-9f7c-4b60-b1e4-1e3569dde6ee None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:08.471891 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1e97c3-9f7c-4b60-b1e4-1e3569dde6ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:08.543725 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: INFO dbcounter [None req-9e1e97c3-9f7c-4b60-b1e4-1e3569dde6ee None None] Registered counter for database keystone Jul 28 17:30:08.565809 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG oslo_db.sqlalchemy.engines [None req-9e1e97c3-9f7c-4b60-b1e4-1e3569dde6ee None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80324) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 28 17:30:08.571921 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writer thread running {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 28 17:30:08.753524 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG passlib.handlers.bcrypt [None req-9e1e97c3-9f7c-4b60-b1e4-1e3569dde6ee None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80324) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 28 17:30:08.753860 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG passlib.handlers.bcrypt [None req-9e1e97c3-9f7c-4b60-b1e4-1e3569dde6ee None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80324) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 28 17:30:08.809407 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-9e1e97c3-9f7c-4b60-b1e4-1e3569dde6ee None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:30:08.811088 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-9e1e97c3-9f7c-4b60-b1e4-1e3569dde6ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:30:08.938312 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 2/3] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:30:08 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 468 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 28 17:30:09.081719 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f61f1e-99e5-45b5-80c9-4f68d1e7bc4f None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:09.152350 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: INFO dbcounter [None req-c3f61f1e-99e5-45b5-80c9-4f68d1e7bc4f None None] Registered counter for database keystone Jul 28 17:30:09.173270 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG oslo_db.sqlalchemy.engines [None req-c3f61f1e-99e5-45b5-80c9-4f68d1e7bc4f None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80325) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 28 17:30:09.179634 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writer thread running {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 28 17:30:09.300621 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f61f1e-99e5-45b5-80c9-4f68d1e7bc4f None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:09.305030 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea64300f-bed3-49c9-a638-2e58db188624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:09.305699 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea64300f-bed3-49c9-a638-2e58db188624 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:09.305986 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea64300f-bed3-49c9-a638-2e58db188624 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:09.306265 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea64300f-bed3-49c9-a638-2e58db188624 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:09.315541 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea64300f-bed3-49c9-a638-2e58db188624 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:09.367569 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: WARNING keystone.common.rbac_enforcer.enforcer [None req-ea64300f-bed3-49c9-a638-2e58db188624 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 28 17:30:09.367845 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea64300f-bed3-49c9-a638-2e58db188624 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:09.378732 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 2/4] 10.222.0.89 () {58 vars in 1324 bytes} [Thu Jul 28 17:30:09 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 28 17:30:09.384675 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1e97c3-9f7c-4b60-b1e4-1e3569dde6ee None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:09.399268 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1e97c3-9f7c-4b60-b1e4-1e3569dde6ee None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:09.403781 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12e6bb5-69fb-4630-862c-d0466779b5bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:09.404869 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12e6bb5-69fb-4630-862c-d0466779b5bb admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:09.405151 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12e6bb5-69fb-4630-862c-d0466779b5bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:09.405456 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12e6bb5-69fb-4630-862c-d0466779b5bb admin admin] PATH_INFO: `/v3/projects/d68024df9ad444518749a9b417c3b67b` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:09.407489 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12e6bb5-69fb-4630-862c-d0466779b5bb admin admin] RBAC: Authorizing `identity:get_project(project_id=d68024df9ad444518749a9b417c3b67b)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:09.458529 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: WARNING keystone.common.rbac_enforcer.enforcer [None req-b12e6bb5-69fb-4630-862c-d0466779b5bb admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 28 17:30:09.458878 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12e6bb5-69fb-4630-862c-d0466779b5bb admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:09.463860 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 3/5] 10.222.0.89 () {56 vars in 1207 bytes} [Thu Jul 28 17:30:09 2022] GET /identity/v3/projects/d68024df9ad444518749a9b417c3b67b => generated 344 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:30:11.460164 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba7194a-02cc-427d-a4e4-04b2a3fb9004 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:11.460836 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba7194a-02cc-427d-a4e4-04b2a3fb9004 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:11.460836 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba7194a-02cc-427d-a4e4-04b2a3fb9004 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:11.464802 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 3/6] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:30:11 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:30:11.470703 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e692bac3-82d9-4dfe-a6c9-272aa5f0469c None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:11.471148 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e692bac3-82d9-4dfe-a6c9-272aa5f0469c None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:11.471245 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e692bac3-82d9-4dfe-a6c9-272aa5f0469c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:11.492626 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-e692bac3-82d9-4dfe-a6c9-272aa5f0469c None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:30:11.494501 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-e692bac3-82d9-4dfe-a6c9-272aa5f0469c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:30:11.502868 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 4/7] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:30:11 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 28 17:30:11.525027 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba7194a-02cc-427d-a4e4-04b2a3fb9004 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:11.537884 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba7194a-02cc-427d-a4e4-04b2a3fb9004 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:11.542075 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765c94b9-bd49-4806-b8a1-ad7a962f2c6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:11.542678 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765c94b9-bd49-4806-b8a1-ad7a962f2c6c admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:11.542964 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765c94b9-bd49-4806-b8a1-ad7a962f2c6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:11.543237 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765c94b9-bd49-4806-b8a1-ad7a962f2c6c admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:11.551851 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765c94b9-bd49-4806-b8a1-ad7a962f2c6c admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:11.553669 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765c94b9-bd49-4806-b8a1-ad7a962f2c6c admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:11.565689 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 4/8] 10.222.0.89 () {58 vars in 1324 bytes} [Thu Jul 28 17:30:11 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 28 17:30:11.571320 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e692bac3-82d9-4dfe-a6c9-272aa5f0469c None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:11.580744 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e692bac3-82d9-4dfe-a6c9-272aa5f0469c None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:11.585256 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795e7b89-c459-4a07-934b-7577a9a4e3c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:11.585889 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795e7b89-c459-4a07-934b-7577a9a4e3c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:11.586261 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795e7b89-c459-4a07-934b-7577a9a4e3c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:11.586544 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795e7b89-c459-4a07-934b-7577a9a4e3c7 admin admin] PATH_INFO: `/v3/users/bbc7ab320041413396df1eef4db728ae` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:11.587918 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795e7b89-c459-4a07-934b-7577a9a4e3c7 admin admin] RBAC: Authorizing `identity:get_user(user_id=bbc7ab320041413396df1eef4db728ae)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:11.589835 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795e7b89-c459-4a07-934b-7577a9a4e3c7 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:11.591372 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 5/9] 10.222.0.89 () {56 vars in 1198 bytes} [Thu Jul 28 17:30:11 2022] GET /identity/v3/users/bbc7ab320041413396df1eef4db728ae => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:30:13.754616 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd41e43-6b41-40bb-b872-64d1c0a7b3b4 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:13.755157 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd41e43-6b41-40bb-b872-64d1c0a7b3b4 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:13.755157 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd41e43-6b41-40bb-b872-64d1c0a7b3b4 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:13.756495 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 5/10] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:30:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:30:13.761802 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de52d5a9-c50f-465b-855c-899fdfedca09 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:13.762230 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de52d5a9-c50f-465b-855c-899fdfedca09 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:13.762375 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de52d5a9-c50f-465b-855c-899fdfedca09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:13.782439 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-de52d5a9-c50f-465b-855c-899fdfedca09 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:30:13.783628 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-de52d5a9-c50f-465b-855c-899fdfedca09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:30:13.800559 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2439ff76-3bf1-4510-9bfb-8337ed9f50d2 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:13.800961 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2439ff76-3bf1-4510-9bfb-8337ed9f50d2 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:13.801386 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2439ff76-3bf1-4510-9bfb-8337ed9f50d2 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:13.802812 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 6/11] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:30:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 17:30:13.807877 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66925d48-2d61-4cfe-8708-0f8619f3be95 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:13.808161 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66925d48-2d61-4cfe-8708-0f8619f3be95 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:13.808435 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66925d48-2d61-4cfe-8708-0f8619f3be95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:13.853341 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG passlib.handlers.bcrypt [None req-66925d48-2d61-4cfe-8708-0f8619f3be95 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80325) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 28 17:30:13.853761 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG passlib.handlers.bcrypt [None req-66925d48-2d61-4cfe-8708-0f8619f3be95 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80325) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 28 17:30:13.867935 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 6/12] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:30:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:30:13.892159 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de52d5a9-c50f-465b-855c-899fdfedca09 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:13.900516 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-66925d48-2d61-4cfe-8708-0f8619f3be95 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:30:13.902126 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-66925d48-2d61-4cfe-8708-0f8619f3be95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:30:13.915696 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:14.002885 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de52d5a9-c50f-465b-855c-899fdfedca09 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:14.005001 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 7/13] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:30:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 17:30:14.027277 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66925d48-2d61-4cfe-8708-0f8619f3be95 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:14.040119 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859ac8a8-16c2-4f51-969b-c83d01dbab74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:14.040881 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859ac8a8-16c2-4f51-969b-c83d01dbab74 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:14.041467 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859ac8a8-16c2-4f51-969b-c83d01dbab74 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:14.041694 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859ac8a8-16c2-4f51-969b-c83d01dbab74 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:14.053852 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859ac8a8-16c2-4f51-969b-c83d01dbab74 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:30:14.063486 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-859ac8a8-16c2-4f51-969b-c83d01dbab74 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:30:14.063486 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:30:14.063486 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:30:14.063486 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:30:14.063486 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:30:14.063486 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:30:14.063486 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:30:14.063486 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:30:14.063486 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:30:14.063486 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:30:14.063486 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:30:14.063486 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:30:14.063486 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:30:14.063486 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:30:14.063486 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:30:14.063486 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:30:14.064442 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:30:14.064442 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:30:14.064442 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:30:14.064442 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:30:14.064442 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:30:14.064442 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:30:14.064442 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:30:14.064442 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:30:14.064442 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:30:14.064442 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:30:14.064442 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:30:14.064442 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( Jul 28 17:30:14.064442 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:30:14.064442 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:30:14.064442 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:30:14.064442 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:30:14.065319 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:30:14.065319 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:30:14.065319 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:30:14.065319 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:30:14.065319 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:30:14.065319 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:30:14.065319 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:30:14.065319 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:30:14.065319 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:30:14.065319 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:30:14.065319 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:30:14.065319 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:30:14.065319 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 7/14] 10.222.0.89 () {56 vars in 1096 bytes} [Thu Jul 28 17:30:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:30:14.070452 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859ac8a8-16c2-4f51-969b-c83d01dbab74 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:14.131052 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66925d48-2d61-4cfe-8708-0f8619f3be95 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:14.133840 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:14.155376 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:14.170029 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078f11c4-7416-4440-83eb-02fc22b71a11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:14.170898 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078f11c4-7416-4440-83eb-02fc22b71a11 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:14.171186 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078f11c4-7416-4440-83eb-02fc22b71a11 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:14.171485 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078f11c4-7416-4440-83eb-02fc22b71a11 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:14.174774 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859ac8a8-16c2-4f51-969b-c83d01dbab74 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:14.208527 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9528f7f5-d95c-43b7-90cf-a645ed2b7790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:14.209142 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9528f7f5-d95c-43b7-90cf-a645ed2b7790 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:14.209459 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9528f7f5-d95c-43b7-90cf-a645ed2b7790 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:14.209736 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9528f7f5-d95c-43b7-90cf-a645ed2b7790 None admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:14.210463 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9528f7f5-d95c-43b7-90cf-a645ed2b7790 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:30:14.210770 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9528f7f5-d95c-43b7-90cf-a645ed2b7790 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:14.212166 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9528f7f5-d95c-43b7-90cf-a645ed2b7790 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:14.224337 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 8/15] 10.222.0.89 () {56 vars in 1099 bytes} [Thu Jul 28 17:30:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:30:14.229709 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9528f7f5-d95c-43b7-90cf-a645ed2b7790 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:14.243220 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078f11c4-7416-4440-83eb-02fc22b71a11 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:14.244998 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078f11c4-7416-4440-83eb-02fc22b71a11 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:14.350690 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 8/16] 10.222.0.89 () {58 vars in 1282 bytes} [Thu Jul 28 17:30:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 17:30:14.357149 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078f11c4-7416-4440-83eb-02fc22b71a11 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:14.362030 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:14.436580 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9528f7f5-d95c-43b7-90cf-a645ed2b7790 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:14.465337 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078f11c4-7416-4440-83eb-02fc22b71a11 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:14.472540 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa41ba9-b2a3-47e3-908b-f14f926ce04d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:14.473213 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa41ba9-b2a3-47e3-908b-f14f926ce04d None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:14.473527 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa41ba9-b2a3-47e3-908b-f14f926ce04d None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:14.473816 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa41ba9-b2a3-47e3-908b-f14f926ce04d None admin] PATH_INFO: `/v3/users/bbc7ab320041413396df1eef4db728ae` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:14.474999 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa41ba9-b2a3-47e3-908b-f14f926ce04d None admin] RBAC: Authorizing `identity:get_user(user_id=bbc7ab320041413396df1eef4db728ae)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:14.476456 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa41ba9-b2a3-47e3-908b-f14f926ce04d None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:14.478479 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 9/17] 10.222.0.89 () {56 vars in 1177 bytes} [Thu Jul 28 17:30:14 2022] GET /identity/v3/users/bbc7ab320041413396df1eef4db728ae => generated 247 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:30:14.483527 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa41ba9-b2a3-47e3-908b-f14f926ce04d None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:14.499431 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:14.502618 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8911be-88f9-442e-bfcb-9b628b7473fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:14.503485 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8911be-88f9-442e-bfcb-9b628b7473fd None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:14.503784 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8911be-88f9-442e-bfcb-9b628b7473fd None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:14.504049 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8911be-88f9-442e-bfcb-9b628b7473fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:14.522606 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8911be-88f9-442e-bfcb-9b628b7473fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:14.524077 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8911be-88f9-442e-bfcb-9b628b7473fd None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:14.542370 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-ae8911be-88f9-442e-bfcb-9b628b7473fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:30:14.542370 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:30:14.542370 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:30:14.542370 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:30:14.542370 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:30:14.542370 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:30:14.542370 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:30:14.542370 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:30:14.542370 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:30:14.542370 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:30:14.542370 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:30:14.542370 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:30:14.542370 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:30:14.542370 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:30:14.542370 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:30:14.543332 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:30:14.543332 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:30:14.543332 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:30:14.543332 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:30:14.543332 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:30:14.543332 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:30:14.543332 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:30:14.543332 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:30:14.543332 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:30:14.543332 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:30:14.543332 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:30:14.543332 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:30:14.543332 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:30:14.543332 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:30:14.543332 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:30:14.543332 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:30:14.543332 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:30:14.544133 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:30:14.544133 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:30:14.544133 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:30:14.544133 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:30:14.544133 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 9/18] 10.222.0.89 () {56 vars in 1108 bytes} [Thu Jul 28 17:30:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:30:14.551272 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8911be-88f9-442e-bfcb-9b628b7473fd None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:14.590520 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa41ba9-b2a3-47e3-908b-f14f926ce04d None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:14.609828 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:14.625745 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3f2517-17b7-4525-a571-9de913067e2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:14.626382 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3f2517-17b7-4525-a571-9de913067e2e None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:14.626665 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3f2517-17b7-4525-a571-9de913067e2e None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:14.626943 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3f2517-17b7-4525-a571-9de913067e2e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:14.628120 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3f2517-17b7-4525-a571-9de913067e2e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:14.629596 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3f2517-17b7-4525-a571-9de913067e2e None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:14.631362 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 10/19] 10.222.0.89 () {56 vars in 1108 bytes} [Thu Jul 28 17:30:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:30:14.636846 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3f2517-17b7-4525-a571-9de913067e2e None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:14.653104 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8911be-88f9-442e-bfcb-9b628b7473fd None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:14.689239 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a999fd-02ac-4c56-8af2-2f67ca14d8d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:14.689869 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a999fd-02ac-4c56-8af2-2f67ca14d8d6 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:14.690151 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a999fd-02ac-4c56-8af2-2f67ca14d8d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:14.690430 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a999fd-02ac-4c56-8af2-2f67ca14d8d6 None admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:14.691138 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a999fd-02ac-4c56-8af2-2f67ca14d8d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:30:14.691446 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a999fd-02ac-4c56-8af2-2f67ca14d8d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:14.692842 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a999fd-02ac-4c56-8af2-2f67ca14d8d6 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:14.710321 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 10/20] 10.222.0.89 () {56 vars in 1109 bytes} [Thu Jul 28 17:30:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:30:14.719713 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:14.741413 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3f2517-17b7-4525-a571-9de913067e2e None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:14.782486 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf96aa0-3a73-4f8c-a686-e25f580ff043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:14.783157 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf96aa0-3a73-4f8c-a686-e25f580ff043 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:14.783387 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf96aa0-3a73-4f8c-a686-e25f580ff043 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:14.783661 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf96aa0-3a73-4f8c-a686-e25f580ff043 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:14.784466 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf96aa0-3a73-4f8c-a686-e25f580ff043 None admin] RBAC: Adding query filter params (role.id=73e457231a824ea98c28d22610be640c, scope.domain.id=default, user.id=bbc7ab320041413396df1eef4db728ae) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:30:14.784775 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf96aa0-3a73-4f8c-a686-e25f580ff043 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:14.786318 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf96aa0-3a73-4f8c-a686-e25f580ff043 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:14.793947 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 11/21] 10.222.0.89 () {56 vars in 1322 bytes} [Thu Jul 28 17:30:14 2022] GET /identity/v3/role_assignments?user.id=bbc7ab320041413396df1eef4db728ae&scope.domain.id=default&role.id=73e457231a824ea98c28d22610be640c => generated 233 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:30:17.217646 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6a443b-c52a-44a0-9162-2cebcbcd697b None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:17.218749 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6a443b-c52a-44a0-9162-2cebcbcd697b None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:17.219446 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6a443b-c52a-44a0-9162-2cebcbcd697b None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:17.221012 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 11/22] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:30:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 17:30:17.226948 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d5b6b9-8062-425c-9393-9d25bae11cc6 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:17.227903 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d5b6b9-8062-425c-9393-9d25bae11cc6 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:17.228382 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d5b6b9-8062-425c-9393-9d25bae11cc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:17.258489 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-c9d5b6b9-8062-425c-9393-9d25bae11cc6 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:30:17.260043 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-c9d5b6b9-8062-425c-9393-9d25bae11cc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:30:17.387482 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 12/23] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:30:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:30:17.422524 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6a443b-c52a-44a0-9162-2cebcbcd697b None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:17.634007 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6a443b-c52a-44a0-9162-2cebcbcd697b None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:17.673531 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:17.729892 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6874cd35-f589-430c-b9d5-6cbe905a8d81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:17.730568 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6874cd35-f589-430c-b9d5-6cbe905a8d81 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:17.730854 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6874cd35-f589-430c-b9d5-6cbe905a8d81 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:17.731132 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6874cd35-f589-430c-b9d5-6cbe905a8d81 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:17.752157 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6874cd35-f589-430c-b9d5-6cbe905a8d81 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:30:17.758798 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-6874cd35-f589-430c-b9d5-6cbe905a8d81 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:30:17.758798 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:30:17.758798 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:30:17.758798 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:30:17.758798 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:30:17.758798 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:30:17.758798 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:30:17.758798 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:30:17.758798 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:30:17.758798 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:30:17.758798 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:30:17.758798 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:30:17.758798 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:30:17.758798 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:30:17.758798 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:30:17.758798 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:30:17.759633 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:30:17.759633 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:30:17.759633 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:30:17.759633 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:30:17.759633 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:30:17.759633 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:30:17.759633 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:30:17.759633 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:30:17.759633 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:30:17.759633 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:30:17.759633 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:30:17.759633 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:30:17.759633 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:30:17.759633 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:30:17.759633 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:30:17.759633 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:30:17.760385 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:30:17.760385 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:30:17.760385 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:30:17.760385 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:30:17.760385 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:30:17.760385 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:30:17.760385 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:30:17.760385 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:30:17.760385 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:30:17.760385 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:30:17.760385 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:30:17.760385 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:30:17.760385 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 12/24] 10.222.0.89 () {56 vars in 1096 bytes} [Thu Jul 28 17:30:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:30:17.766789 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d5b6b9-8062-425c-9393-9d25bae11cc6 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:17.851461 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a179816e-e584-4760-a68e-b0ddf0e1baaa None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:17.851994 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a179816e-e584-4760-a68e-b0ddf0e1baaa None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:17.852423 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a179816e-e584-4760-a68e-b0ddf0e1baaa None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:17.853940 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 13/25] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:30:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 17:30:17.858694 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748b188a-1c8f-4f96-9059-2efa4de8bb10 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:17.859222 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748b188a-1c8f-4f96-9059-2efa4de8bb10 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:17.859873 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748b188a-1c8f-4f96-9059-2efa4de8bb10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:17.889525 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-748b188a-1c8f-4f96-9059-2efa4de8bb10 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:30:17.891323 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-748b188a-1c8f-4f96-9059-2efa4de8bb10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:30:17.961837 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:17.990608 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 14/26] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:30:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 17:30:17.991007 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d5b6b9-8062-425c-9393-9d25bae11cc6 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:17.992120 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b657f521-d3bd-4e88-9a3d-56c0d3effc64 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:17.992506 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b657f521-d3bd-4e88-9a3d-56c0d3effc64 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:17.993257 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b657f521-d3bd-4e88-9a3d-56c0d3effc64 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:17.995913 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 15/27] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:30:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:30:17.997924 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d522e3-ab66-47c5-9d4c-77fad12f6752 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:18.001522 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d522e3-ab66-47c5-9d4c-77fad12f6752 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:18.001999 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d522e3-ab66-47c5-9d4c-77fad12f6752 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:18.010362 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 16/28] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:30:17 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:30:18.011972 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c75cef6-d9fa-4cd2-8c79-da631049d19a None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:18.013671 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c75cef6-d9fa-4cd2-8c79-da631049d19a None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:18.014120 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c75cef6-d9fa-4cd2-8c79-da631049d19a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:18.045781 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-1c75cef6-d9fa-4cd2-8c79-da631049d19a None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:30:18.047385 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-1c75cef6-d9fa-4cd2-8c79-da631049d19a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:30:18.049307 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b424187-2244-4c94-9e01-c1b1bd03b097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:18.050404 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b424187-2244-4c94-9e01-c1b1bd03b097 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:18.051869 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b424187-2244-4c94-9e01-c1b1bd03b097 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:18.052301 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b424187-2244-4c94-9e01-c1b1bd03b097 None admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:18.053188 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b424187-2244-4c94-9e01-c1b1bd03b097 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:30:18.053658 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b424187-2244-4c94-9e01-c1b1bd03b097 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:18.055271 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b424187-2244-4c94-9e01-c1b1bd03b097 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:18.068151 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 13/29] 10.222.0.89 () {56 vars in 1099 bytes} [Thu Jul 28 17:30:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:30:18.070321 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe54d55-76e0-4708-9b11-eaceec241f37 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:18.070756 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe54d55-76e0-4708-9b11-eaceec241f37 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:18.071203 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe54d55-76e0-4708-9b11-eaceec241f37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:18.095918 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-1fe54d55-76e0-4708-9b11-eaceec241f37 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:30:18.098045 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-1fe54d55-76e0-4708-9b11-eaceec241f37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:30:18.178412 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 14/30] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:30:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:30:18.179574 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe54d55-76e0-4708-9b11-eaceec241f37 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:18.257181 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 17/31] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:30:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:30:18.257181 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c75cef6-d9fa-4cd2-8c79-da631049d19a None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:18.268363 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:18.300237 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:18.305228 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe54d55-76e0-4708-9b11-eaceec241f37 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:18.343061 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d221af5b-f4fd-4395-9a7e-599cb8360ed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:18.343715 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d221af5b-f4fd-4395-9a7e-599cb8360ed2 None admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:18.343997 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d221af5b-f4fd-4395-9a7e-599cb8360ed2 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:18.344447 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d221af5b-f4fd-4395-9a7e-599cb8360ed2 None admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:18.344985 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d221af5b-f4fd-4395-9a7e-599cb8360ed2 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:18.347467 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d221af5b-f4fd-4395-9a7e-599cb8360ed2 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:18.379835 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c75cef6-d9fa-4cd2-8c79-da631049d19a None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:18.416669 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1583d06-2aa9-4977-8019-011a7ddd2a98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:18.417354 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1583d06-2aa9-4977-8019-011a7ddd2a98 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:18.417638 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1583d06-2aa9-4977-8019-011a7ddd2a98 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:18.417915 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1583d06-2aa9-4977-8019-011a7ddd2a98 None admin] PATH_INFO: `/v3/users/bbc7ab320041413396df1eef4db728ae` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:18.419460 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1583d06-2aa9-4977-8019-011a7ddd2a98 None admin] RBAC: Authorizing `identity:get_user(user_id=bbc7ab320041413396df1eef4db728ae)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:18.420951 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1583d06-2aa9-4977-8019-011a7ddd2a98 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:18.427383 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 18/32] 10.222.0.89 () {56 vars in 1177 bytes} [Thu Jul 28 17:30:18 2022] GET /identity/v3/users/bbc7ab320041413396df1eef4db728ae => generated 247 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:30:18.427383 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20458dd0-317c-42cb-8108-47d640785f33 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:18.427383 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20458dd0-317c-42cb-8108-47d640785f33 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:18.427383 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20458dd0-317c-42cb-8108-47d640785f33 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:18.427383 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 19/33] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:30:18 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:30:18.427383 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20458dd0-317c-42cb-8108-47d640785f33 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:18.471447 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 15/34] 10.222.0.89 () {60 vars in 1131 bytes} [Thu Jul 28 17:30:18 2022] POST /identity/v3/roles => generated 219 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:30:18.472194 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d221af5b-f4fd-4395-9a7e-599cb8360ed2 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:18.558337 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20458dd0-317c-42cb-8108-47d640785f33 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:18.561361 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:18.597489 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100d9900-88b7-499d-b48f-5d4362b13709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:18.598649 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100d9900-88b7-499d-b48f-5d4362b13709 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:18.598942 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100d9900-88b7-499d-b48f-5d4362b13709 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:18.599221 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100d9900-88b7-499d-b48f-5d4362b13709 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:18.611585 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100d9900-88b7-499d-b48f-5d4362b13709 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:18.613080 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100d9900-88b7-499d-b48f-5d4362b13709 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:18.614193 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d221af5b-f4fd-4395-9a7e-599cb8360ed2 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:18.630282 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-100d9900-88b7-499d-b48f-5d4362b13709 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:30:18.630282 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:30:18.630282 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:30:18.630282 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:30:18.630282 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:30:18.630282 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:30:18.630282 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:30:18.630282 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:30:18.630282 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:30:18.630282 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:30:18.630282 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:30:18.630282 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:30:18.630282 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:30:18.630282 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:30:18.630282 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:30:18.631371 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:30:18.631371 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:30:18.631371 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:30:18.631371 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:30:18.631371 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:30:18.631371 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:30:18.631371 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:30:18.631371 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:30:18.631371 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:30:18.631371 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:30:18.631371 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:30:18.631371 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:30:18.631371 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:30:18.631371 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:30:18.631371 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:30:18.631371 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:30:18.631371 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:30:18.632378 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:30:18.632378 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:30:18.632378 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:30:18.632378 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:30:18.632378 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 20/35] 10.222.0.89 () {56 vars in 1108 bytes} [Thu Jul 28 17:30:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:30:18.632378 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100d9900-88b7-499d-b48f-5d4362b13709 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:18.656491 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:18.671879 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b202ff1-8240-4f9b-a939-81ba4e55f432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:18.672823 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b202ff1-8240-4f9b-a939-81ba4e55f432 None admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:18.673271 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b202ff1-8240-4f9b-a939-81ba4e55f432 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:18.673561 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b202ff1-8240-4f9b-a939-81ba4e55f432 None admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:18.674350 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b202ff1-8240-4f9b-a939-81ba4e55f432 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:18.675907 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b202ff1-8240-4f9b-a939-81ba4e55f432 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:18.696237 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 16/36] 10.222.0.89 () {60 vars in 1131 bytes} [Thu Jul 28 17:30:18 2022] POST /identity/v3/roles => generated 225 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:30:18.697600 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd22749a-7287-4209-922a-a651355bc6ca None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:18.697965 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd22749a-7287-4209-922a-a651355bc6ca None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:18.698318 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd22749a-7287-4209-922a-a651355bc6ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:18.719407 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-fd22749a-7287-4209-922a-a651355bc6ca None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:30:18.721271 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-fd22749a-7287-4209-922a-a651355bc6ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:30:18.770899 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100d9900-88b7-499d-b48f-5d4362b13709 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:18.810684 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 17/37] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:30:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:30:18.812524 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6e1981-fc66-4af5-9e48-9e00056cd727 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:18.813094 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6e1981-fc66-4af5-9e48-9e00056cd727 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:18.813613 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6e1981-fc66-4af5-9e48-9e00056cd727 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:18.814993 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 18/38] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:30:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:30:18.815738 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6e1981-fc66-4af5-9e48-9e00056cd727 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:18.850480 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ce6963-bb3e-4c26-8bd7-66a006ea1dd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:18.851140 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ce6963-bb3e-4c26-8bd7-66a006ea1dd8 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:18.851416 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ce6963-bb3e-4c26-8bd7-66a006ea1dd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:18.854357 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ce6963-bb3e-4c26-8bd7-66a006ea1dd8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:18.854357 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ce6963-bb3e-4c26-8bd7-66a006ea1dd8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:18.854829 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ce6963-bb3e-4c26-8bd7-66a006ea1dd8 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:18.856609 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 21/39] 10.222.0.89 () {56 vars in 1108 bytes} [Thu Jul 28 17:30:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:30:18.857682 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fe35fa-b5be-4829-b165-96cddee80387 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:18.858107 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fe35fa-b5be-4829-b165-96cddee80387 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:18.858383 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fe35fa-b5be-4829-b165-96cddee80387 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:18.879092 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-e3fe35fa-b5be-4829-b165-96cddee80387 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:30:18.880308 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-e3fe35fa-b5be-4829-b165-96cddee80387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:30:18.907268 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:18.944803 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6e1981-fc66-4af5-9e48-9e00056cd727 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:18.960439 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 22/40] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:30:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:30:18.961197 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fe35fa-b5be-4829-b165-96cddee80387 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:18.963018 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:18.985874 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c58b2c-dbca-4fea-bb59-7a170127fa86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:18.986751 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c58b2c-dbca-4fea-bb59-7a170127fa86 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:18.987159 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c58b2c-dbca-4fea-bb59-7a170127fa86 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:18.987573 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c58b2c-dbca-4fea-bb59-7a170127fa86 None admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:18.988753 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c58b2c-dbca-4fea-bb59-7a170127fa86 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:30:18.989068 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c58b2c-dbca-4fea-bb59-7a170127fa86 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:18.991273 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c58b2c-dbca-4fea-bb59-7a170127fa86 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:19.012898 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 19/41] 10.222.0.89 () {56 vars in 1109 bytes} [Thu Jul 28 17:30:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:30:19.013778 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c58b2c-dbca-4fea-bb59-7a170127fa86 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:19.065750 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fe35fa-b5be-4829-b165-96cddee80387 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:19.101763 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b7dcf8-2d9b-49de-8ff9-a020aeaa856c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:19.102392 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b7dcf8-2d9b-49de-8ff9-a020aeaa856c None admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:19.102673 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b7dcf8-2d9b-49de-8ff9-a020aeaa856c None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:19.102947 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b7dcf8-2d9b-49de-8ff9-a020aeaa856c None admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:19.103668 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b7dcf8-2d9b-49de-8ff9-a020aeaa856c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:19.105048 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b7dcf8-2d9b-49de-8ff9-a020aeaa856c None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:19.122245 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c58b2c-dbca-4fea-bb59-7a170127fa86 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:19.157870 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5846e1a-a9cd-465c-a22f-22c48582339b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:19.158546 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5846e1a-a9cd-465c-a22f-22c48582339b None admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:19.158834 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5846e1a-a9cd-465c-a22f-22c48582339b None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:19.159117 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5846e1a-a9cd-465c-a22f-22c48582339b None admin] PATH_INFO: `/v3/domains/default/users/bbc7ab320041413396df1eef4db728ae/roles/73e457231a824ea98c28d22610be640c` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:19.161140 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5846e1a-a9cd-465c-a22f-22c48582339b None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=bbc7ab320041413396df1eef4db728ae, role_id=73e457231a824ea98c28d22610be640c)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:19.162685 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5846e1a-a9cd-465c-a22f-22c48582339b None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:19.176716 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 20/42] 10.222.0.89 () {58 vars in 1361 bytes} [Thu Jul 28 17:30:19 2022] PUT /identity/v3/domains/default/users/bbc7ab320041413396df1eef4db728ae/roles/73e457231a824ea98c28d22610be640c => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:30:19.177581 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5846e1a-a9cd-465c-a22f-22c48582339b None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:19.222672 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:19.229947 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 23/43] 10.222.0.89 () {60 vars in 1131 bytes} [Thu Jul 28 17:30:18 2022] POST /identity/v3/roles => generated 223 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:30:19.230983 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b7dcf8-2d9b-49de-8ff9-a020aeaa856c None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:19.254065 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:19.317020 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5846e1a-a9cd-465c-a22f-22c48582339b None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:19.362600 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b7dcf8-2d9b-49de-8ff9-a020aeaa856c None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:19.374028 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6d4e05-d91f-4532-a9bf-f61c618fb992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:19.376915 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6d4e05-d91f-4532-a9bf-f61c618fb992 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:19.377249 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6d4e05-d91f-4532-a9bf-f61c618fb992 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:19.377530 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6d4e05-d91f-4532-a9bf-f61c618fb992 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:19.379334 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6d4e05-d91f-4532-a9bf-f61c618fb992 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:19.381122 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6d4e05-d91f-4532-a9bf-f61c618fb992 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:19.382890 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 21/44] 10.222.0.89 () {56 vars in 1108 bytes} [Thu Jul 28 17:30:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:30:19.388587 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6d4e05-d91f-4532-a9bf-f61c618fb992 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:19.426423 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15517adc-d8a1-4b73-a2f6-37a9e050fe34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:19.427083 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15517adc-d8a1-4b73-a2f6-37a9e050fe34 None admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:19.427373 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15517adc-d8a1-4b73-a2f6-37a9e050fe34 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:19.427654 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15517adc-d8a1-4b73-a2f6-37a9e050fe34 None admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:19.428367 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15517adc-d8a1-4b73-a2f6-37a9e050fe34 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:19.430089 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15517adc-d8a1-4b73-a2f6-37a9e050fe34 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:19.471562 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 24/45] 10.222.0.89 () {60 vars in 1141 bytes} [Thu Jul 28 17:30:19 2022] POST /identity/v3/projects => generated 301 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:30:19.520291 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6d4e05-d91f-4532-a9bf-f61c618fb992 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:19.523946 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:19.565057 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a938055d-2304-4deb-8b14-6e2b3b1e3c42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:19.565736 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a938055d-2304-4deb-8b14-6e2b3b1e3c42 None admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:19.566020 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a938055d-2304-4deb-8b14-6e2b3b1e3c42 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:19.566294 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a938055d-2304-4deb-8b14-6e2b3b1e3c42 None admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:19.567005 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a938055d-2304-4deb-8b14-6e2b3b1e3c42 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:19.568556 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a938055d-2304-4deb-8b14-6e2b3b1e3c42 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:19.599967 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 22/46] 10.222.0.89 () {60 vars in 1141 bytes} [Thu Jul 28 17:30:19 2022] POST /identity/v3/projects => generated 312 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 17:30:21.271204 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e00e838-67d9-4d05-951a-d9fdb88d509b None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:21.271714 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e00e838-67d9-4d05-951a-d9fdb88d509b None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:21.271797 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e00e838-67d9-4d05-951a-d9fdb88d509b None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:21.273115 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 25/47] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:30:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:30:21.278238 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ba2888-74a3-448a-bf15-47512481fbf6 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:21.278645 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ba2888-74a3-448a-bf15-47512481fbf6 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:21.278796 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ba2888-74a3-448a-bf15-47512481fbf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:21.298801 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-f2ba2888-74a3-448a-bf15-47512481fbf6 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:30:21.300001 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-f2ba2888-74a3-448a-bf15-47512481fbf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:30:21.371108 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 23/48] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:30:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:30:21.395177 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e00e838-67d9-4d05-951a-d9fdb88d509b None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:21.463092 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:21.499233 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e00e838-67d9-4d05-951a-d9fdb88d509b None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:21.533039 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd271e84-628f-4498-8761-6c3c158f5809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:21.534273 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd271e84-628f-4498-8761-6c3c158f5809 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:21.534273 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd271e84-628f-4498-8761-6c3c158f5809 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:21.534273 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd271e84-628f-4498-8761-6c3c158f5809 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:21.545393 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd271e84-628f-4498-8761-6c3c158f5809 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:30:21.551113 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-dd271e84-628f-4498-8761-6c3c158f5809 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:30:21.551113 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:30:21.551113 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:30:21.551113 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:30:21.551113 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:30:21.551113 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:30:21.551113 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:30:21.551113 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:30:21.551113 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:30:21.551113 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:30:21.551113 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:30:21.551113 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:30:21.551113 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:30:21.551113 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:30:21.551113 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:30:21.551113 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:30:21.552116 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:30:21.552116 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:30:21.552116 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:30:21.552116 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:30:21.552116 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:30:21.552116 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:30:21.552116 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:30:21.552116 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:30:21.552116 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:30:21.552116 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:30:21.552116 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:30:21.552116 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:30:21.552116 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:30:21.552116 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:30:21.552116 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:30:21.552116 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:30:21.552992 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:30:21.552992 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:30:21.552992 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:30:21.552992 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:30:21.552992 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:30:21.552992 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:30:21.552992 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:30:21.552992 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:30:21.552992 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:30:21.552992 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:30:21.552992 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:30:21.552992 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:30:21.552992 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 26/49] 10.222.0.89 () {56 vars in 1096 bytes} [Thu Jul 28 17:30:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:30:21.558842 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ba2888-74a3-448a-bf15-47512481fbf6 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:21.621562 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f806ab6-6dc0-4c94-9b36-49f516ac20bd None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:21.621843 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f806ab6-6dc0-4c94-9b36-49f516ac20bd None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:21.622141 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f806ab6-6dc0-4c94-9b36-49f516ac20bd None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:21.623486 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 27/50] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:30:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 17:30:21.629640 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d2c09f-9af4-4051-880f-13cd7779befb None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:21.629964 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d2c09f-9af4-4051-880f-13cd7779befb None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:21.630268 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d2c09f-9af4-4051-880f-13cd7779befb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:21.647031 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:21.648828 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-e7d2c09f-9af4-4051-880f-13cd7779befb None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:30:21.650750 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-e7d2c09f-9af4-4051-880f-13cd7779befb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:30:21.663267 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ba2888-74a3-448a-bf15-47512481fbf6 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:21.697720 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa4f9fa-473e-49f1-a9a2-f771839b23bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:21.698341 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa4f9fa-473e-49f1-a9a2-f771839b23bb None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:21.698620 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa4f9fa-473e-49f1-a9a2-f771839b23bb None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:21.698897 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa4f9fa-473e-49f1-a9a2-f771839b23bb None admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:21.699615 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa4f9fa-473e-49f1-a9a2-f771839b23bb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:30:21.699918 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa4f9fa-473e-49f1-a9a2-f771839b23bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:21.701333 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa4f9fa-473e-49f1-a9a2-f771839b23bb None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:21.709586 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 24/51] 10.222.0.89 () {56 vars in 1099 bytes} [Thu Jul 28 17:30:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:30:21.715236 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa4f9fa-473e-49f1-a9a2-f771839b23bb None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:21.720382 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 28/52] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:30:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 17:30:21.743495 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d2c09f-9af4-4051-880f-13cd7779befb None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:21.800144 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:21.819827 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa4f9fa-473e-49f1-a9a2-f771839b23bb None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:21.845380 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d2c09f-9af4-4051-880f-13cd7779befb None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:21.853687 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a925d00-9ac2-492d-bfb6-fcf072a23525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:21.854322 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a925d00-9ac2-492d-bfb6-fcf072a23525 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:21.854601 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a925d00-9ac2-492d-bfb6-fcf072a23525 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:21.854872 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a925d00-9ac2-492d-bfb6-fcf072a23525 None admin] PATH_INFO: `/v3/users/bbc7ab320041413396df1eef4db728ae` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:21.856197 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a925d00-9ac2-492d-bfb6-fcf072a23525 None admin] RBAC: Authorizing `identity:get_user(user_id=bbc7ab320041413396df1eef4db728ae)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:21.857672 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a925d00-9ac2-492d-bfb6-fcf072a23525 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:21.860314 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 25/53] 10.222.0.89 () {56 vars in 1177 bytes} [Thu Jul 28 17:30:21 2022] GET /identity/v3/users/bbc7ab320041413396df1eef4db728ae => generated 247 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:30:21.865612 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a925d00-9ac2-492d-bfb6-fcf072a23525 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:21.880723 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b074f63b-7380-499c-a8ef-1bff89f0b022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:21.881486 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b074f63b-7380-499c-a8ef-1bff89f0b022 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:21.881773 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b074f63b-7380-499c-a8ef-1bff89f0b022 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:21.882050 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b074f63b-7380-499c-a8ef-1bff89f0b022 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:21.883218 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b074f63b-7380-499c-a8ef-1bff89f0b022 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:21.884659 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b074f63b-7380-499c-a8ef-1bff89f0b022 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:21.886286 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 29/54] 10.222.0.89 () {56 vars in 1108 bytes} [Thu Jul 28 17:30:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:30:21.886924 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:21.891917 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b074f63b-7380-499c-a8ef-1bff89f0b022 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:21.969662 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a925d00-9ac2-492d-bfb6-fcf072a23525 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:21.992050 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b074f63b-7380-499c-a8ef-1bff89f0b022 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:22.005018 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e21e655-ba81-4f7c-b1b4-b50dd83b45be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:22.005686 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e21e655-ba81-4f7c-b1b4-b50dd83b45be None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:22.005966 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e21e655-ba81-4f7c-b1b4-b50dd83b45be None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:22.006247 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e21e655-ba81-4f7c-b1b4-b50dd83b45be None admin] PATH_INFO: `/v3/domains/default` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:22.007397 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e21e655-ba81-4f7c-b1b4-b50dd83b45be None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:22.008829 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e21e655-ba81-4f7c-b1b4-b50dd83b45be None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:22.010447 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 26/55] 10.222.0.89 () {56 vars in 1108 bytes} [Thu Jul 28 17:30:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:30:22.016012 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e21e655-ba81-4f7c-b1b4-b50dd83b45be None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:22.016351 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:22.027511 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c2ae92-2097-4449-8a4b-ecfdf5371ee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:22.028143 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c2ae92-2097-4449-8a4b-ecfdf5371ee6 None admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:22.028422 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c2ae92-2097-4449-8a4b-ecfdf5371ee6 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:22.028706 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c2ae92-2097-4449-8a4b-ecfdf5371ee6 None admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:22.029410 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c2ae92-2097-4449-8a4b-ecfdf5371ee6 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:22.031741 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c2ae92-2097-4449-8a4b-ecfdf5371ee6 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:22.058469 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 30/56] 10.222.0.89 () {60 vars in 1140 bytes} [Thu Jul 28 17:30:21 2022] POST /identity/v3/projects => generated 298 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 17:30:22.108696 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:22.127490 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e21e655-ba81-4f7c-b1b4-b50dd83b45be None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:22.163448 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61619005-18a9-4243-8ff6-ac8fa27472d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:22.164077 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61619005-18a9-4243-8ff6-ac8fa27472d3 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:22.164363 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61619005-18a9-4243-8ff6-ac8fa27472d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:22.164644 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61619005-18a9-4243-8ff6-ac8fa27472d3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:22.165493 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61619005-18a9-4243-8ff6-ac8fa27472d3 None admin] RBAC: Adding query filter params (role.id=73e457231a824ea98c28d22610be640c, scope.domain.id=default, user.id=bbc7ab320041413396df1eef4db728ae) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:30:22.165809 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61619005-18a9-4243-8ff6-ac8fa27472d3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:22.167320 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61619005-18a9-4243-8ff6-ac8fa27472d3 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:22.174722 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 27/57] 10.222.0.89 () {56 vars in 1322 bytes} [Thu Jul 28 17:30:22 2022] GET /identity/v3/role_assignments?user.id=bbc7ab320041413396df1eef4db728ae&scope.domain.id=default&role.id=73e457231a824ea98c28d22610be640c => generated 530 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:30:24.108148 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0c901b-3a30-40ee-b8a6-0a3f8ee9b01a None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:24.109215 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0c901b-3a30-40ee-b8a6-0a3f8ee9b01a None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:24.109215 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0c901b-3a30-40ee-b8a6-0a3f8ee9b01a None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:24.110600 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 31/58] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:30:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:30:24.115812 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38927611-d9eb-4da1-90f9-3de5319eb9ec None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:24.116214 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38927611-d9eb-4da1-90f9-3de5319eb9ec None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:24.116358 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38927611-d9eb-4da1-90f9-3de5319eb9ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:24.135716 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-38927611-d9eb-4da1-90f9-3de5319eb9ec None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:30:24.136910 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-38927611-d9eb-4da1-90f9-3de5319eb9ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:30:24.201653 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 28/59] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:30:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:30:24.226649 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0c901b-3a30-40ee-b8a6-0a3f8ee9b01a None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:24.326116 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0c901b-3a30-40ee-b8a6-0a3f8ee9b01a None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:24.360888 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb704475-c8a4-4168-9871-1e1906c90702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:24.361493 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb704475-c8a4-4168-9871-1e1906c90702 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:24.361771 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb704475-c8a4-4168-9871-1e1906c90702 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:24.362044 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb704475-c8a4-4168-9871-1e1906c90702 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:24.363200 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb704475-c8a4-4168-9871-1e1906c90702 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:24.364648 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb704475-c8a4-4168-9871-1e1906c90702 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:24.366560 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 32/60] 10.222.0.89 () {56 vars in 1108 bytes} [Thu Jul 28 17:30:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:30:24.372367 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38927611-d9eb-4da1-90f9-3de5319eb9ec None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:24.451105 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:24.473809 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38927611-d9eb-4da1-90f9-3de5319eb9ec None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:24.509272 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31984c0a-6bc2-4e74-8d2b-87f4029de71b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:24.509939 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31984c0a-6bc2-4e74-8d2b-87f4029de71b None admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:24.510176 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31984c0a-6bc2-4e74-8d2b-87f4029de71b None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:24.510455 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31984c0a-6bc2-4e74-8d2b-87f4029de71b None admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:24.511157 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31984c0a-6bc2-4e74-8d2b-87f4029de71b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:24.512604 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31984c0a-6bc2-4e74-8d2b-87f4029de71b None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:24.541213 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 29/61] 10.222.0.89 () {60 vars in 1132 bytes} [Thu Jul 28 17:30:24 2022] POST /identity/v3/users => generated 275 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:30:27.903476 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e1e9c8-7e7b-4874-a659-172be5ef1bab None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:27.903936 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e1e9c8-7e7b-4874-a659-172be5ef1bab None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:27.903936 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e1e9c8-7e7b-4874-a659-172be5ef1bab None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:27.905462 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 33/62] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:30:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:30:27.912029 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c86e94d-f738-499a-9eb8-d6e2e087e5f2 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:27.912482 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c86e94d-f738-499a-9eb8-d6e2e087e5f2 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:27.912655 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c86e94d-f738-499a-9eb8-d6e2e087e5f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:27.937696 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf91503-8230-4dfd-b0aa-ebb0eb6810fc None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:27.938328 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf91503-8230-4dfd-b0aa-ebb0eb6810fc None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:27.938832 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf91503-8230-4dfd-b0aa-ebb0eb6810fc None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:27.940690 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 34/63] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:30:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:30:27.945737 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-0c86e94d-f738-499a-9eb8-d6e2e087e5f2 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:30:27.946879 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af09cb10-2159-494a-9d66-d4a1fcaa0533 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:27.947004 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-0c86e94d-f738-499a-9eb8-d6e2e087e5f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:30:27.947499 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af09cb10-2159-494a-9d66-d4a1fcaa0533 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:27.947978 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af09cb10-2159-494a-9d66-d4a1fcaa0533 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:27.989515 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-af09cb10-2159-494a-9d66-d4a1fcaa0533 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:30:27.991132 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-af09cb10-2159-494a-9d66-d4a1fcaa0533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:30:28.068568 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 30/64] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:30:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:30:28.094147 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:28.114250 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c86e94d-f738-499a-9eb8-d6e2e087e5f2 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:28.189012 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 35/65] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:30:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:30:28.190382 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc6f705-b44d-48b0-b772-4a8f5ca3afc7 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:28.193211 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc6f705-b44d-48b0-b772-4a8f5ca3afc7 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:28.193211 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc6f705-b44d-48b0-b772-4a8f5ca3afc7 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:28.196315 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 36/66] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:30:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:30:28.214997 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7646934-c479-4408-a21d-fb826e4d7316 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:28.215304 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7646934-c479-4408-a21d-fb826e4d7316 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:28.215544 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7646934-c479-4408-a21d-fb826e4d7316 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:28.219873 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:28.251430 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-e7646934-c479-4408-a21d-fb826e4d7316 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:30:28.254020 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-e7646934-c479-4408-a21d-fb826e4d7316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:30:28.275964 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c86e94d-f738-499a-9eb8-d6e2e087e5f2 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:28.321014 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119a648d-58c5-4086-80a0-a988b7e9976d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:28.321697 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119a648d-58c5-4086-80a0-a988b7e9976d None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:28.321978 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119a648d-58c5-4086-80a0-a988b7e9976d None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:28.322256 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119a648d-58c5-4086-80a0-a988b7e9976d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:28.339046 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119a648d-58c5-4086-80a0-a988b7e9976d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:30:28.346989 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-119a648d-58c5-4086-80a0-a988b7e9976d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 17:30:28.346989 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:30:28.346989 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:30:28.346989 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:30:28.346989 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:30:28.346989 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:30:28.346989 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:30:28.346989 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:30:28.346989 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:30:28.346989 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:30:28.346989 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:30:28.346989 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:30:28.346989 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:30:28.346989 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:30:28.346989 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:30:28.346989 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:30:28.347955 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:30:28.347955 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:30:28.347955 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:30:28.347955 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:30:28.347955 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:30:28.347955 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:30:28.347955 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:30:28.347955 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:30:28.347955 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:30:28.347955 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:30:28.347955 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:30:28.347955 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( Jul 28 17:30:28.347955 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:30:28.347955 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:30:28.347955 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:30:28.347955 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:30:28.348715 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:30:28.348715 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:30:28.348715 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:30:28.348715 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:30:28.348715 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:30:28.348715 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:30:28.348715 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:30:28.348715 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:30:28.348715 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:30:28.348715 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:30:28.348715 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 17:30:28.348715 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:30:28.365805 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 31/67] 10.222.0.89 () {56 vars in 1099 bytes} [Thu Jul 28 17:30:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:30:28.367025 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119a648d-58c5-4086-80a0-a988b7e9976d None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:28.418686 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 37/68] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:30:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 17:30:28.420522 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7646934-c479-4408-a21d-fb826e4d7316 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:28.504487 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119a648d-58c5-4086-80a0-a988b7e9976d None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:28.529742 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:28.542823 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b99190-7206-47bb-8c48-46af3da9b766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:28.543400 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b99190-7206-47bb-8c48-46af3da9b766 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:28.543786 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b99190-7206-47bb-8c48-46af3da9b766 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:28.544075 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b99190-7206-47bb-8c48-46af3da9b766 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:28.554746 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b99190-7206-47bb-8c48-46af3da9b766 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:30:28.555135 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7646934-c479-4408-a21d-fb826e4d7316 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:28.557859 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:28.559821 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-b9b99190-7206-47bb-8c48-46af3da9b766 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:30:28.559821 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:30:28.559821 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:30:28.559821 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:30:28.559821 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:30:28.559821 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:30:28.559821 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:30:28.559821 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:30:28.559821 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:30:28.559821 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:30:28.559821 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:30:28.559821 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:30:28.559821 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:30:28.559821 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:30:28.559821 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:30:28.559821 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:30:28.561064 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:30:28.561064 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:30:28.561064 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:30:28.561064 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:30:28.561064 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:30:28.561064 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:30:28.561064 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:30:28.561064 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:30:28.561064 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:30:28.561064 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:30:28.561064 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:30:28.561064 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( Jul 28 17:30:28.561064 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:30:28.561064 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:30:28.561064 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:30:28.561064 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:30:28.562285 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:30:28.562285 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:30:28.562285 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:30:28.562285 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:30:28.562285 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:30:28.562285 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:30:28.562285 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:30:28.562285 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:30:28.562285 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:30:28.562285 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:30:28.562285 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:30:28.562285 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:30:28.562285 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 32/69] 10.222.0.89 () {56 vars in 1096 bytes} [Thu Jul 28 17:30:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:30:28.562285 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd4f58e-0c6d-470b-baee-5ff5fc106ce7 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:28.562972 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd4f58e-0c6d-470b-baee-5ff5fc106ce7 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:28.562972 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd4f58e-0c6d-470b-baee-5ff5fc106ce7 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:28.565196 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 33/70] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:30:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:30:28.565196 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd4f58e-0c6d-470b-baee-5ff5fc106ce7 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:28.602020 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcbe303-21a4-4b03-bc38-2f2c6f69e435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:28.602573 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcbe303-21a4-4b03-bc38-2f2c6f69e435 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:28.602858 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcbe303-21a4-4b03-bc38-2f2c6f69e435 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:28.603135 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcbe303-21a4-4b03-bc38-2f2c6f69e435 None admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:28.603864 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcbe303-21a4-4b03-bc38-2f2c6f69e435 None admin] RBAC: Adding query filter params (name=member) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:30:28.604172 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcbe303-21a4-4b03-bc38-2f2c6f69e435 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:28.605600 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcbe303-21a4-4b03-bc38-2f2c6f69e435 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:28.617870 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 38/71] 10.222.0.89 () {56 vars in 1101 bytes} [Thu Jul 28 17:30:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:30:28.619379 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6c1f82-36ad-4004-91c6-c1efb7a5c11e None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:28.619667 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6c1f82-36ad-4004-91c6-c1efb7a5c11e None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:28.619947 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6c1f82-36ad-4004-91c6-c1efb7a5c11e None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:28.625278 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 39/72] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:30:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:30:28.626329 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6c1f82-36ad-4004-91c6-c1efb7a5c11e None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:28.699053 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd4f58e-0c6d-470b-baee-5ff5fc106ce7 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:28.741785 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f3d016-094a-4cb4-ac71-148b06bd3eec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:28.742625 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f3d016-094a-4cb4-ac71-148b06bd3eec None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:28.743105 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f3d016-094a-4cb4-ac71-148b06bd3eec None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:28.743447 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f3d016-094a-4cb4-ac71-148b06bd3eec None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:28.745699 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6c1f82-36ad-4004-91c6-c1efb7a5c11e None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:28.757205 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f3d016-094a-4cb4-ac71-148b06bd3eec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:30:28.762247 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-d0f3d016-094a-4cb4-ac71-148b06bd3eec None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 17:30:28.762247 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:30:28.762247 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:30:28.762247 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:30:28.762247 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:30:28.762247 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:30:28.762247 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:30:28.762247 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:30:28.762247 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:30:28.762247 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:30:28.762247 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:30:28.762247 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:30:28.762247 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:30:28.762247 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:30:28.762247 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:30:28.762247 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:30:28.763086 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:30:28.763086 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:30:28.763086 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:30:28.763086 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:30:28.763086 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:30:28.763086 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:30:28.763086 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:30:28.763086 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:30:28.763086 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:30:28.763086 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:30:28.763086 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:30:28.763086 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( Jul 28 17:30:28.763086 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:30:28.763086 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:30:28.763086 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:30:28.763086 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:30:28.763874 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:30:28.763874 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:30:28.763874 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:30:28.763874 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:30:28.763874 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:30:28.763874 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:30:28.763874 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:30:28.763874 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:30:28.763874 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:30:28.763874 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:30:28.763874 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 17:30:28.763874 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:30:28.763874 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 34/73] 10.222.0.89 () {56 vars in 1114 bytes} [Thu Jul 28 17:30:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:30:28.764782 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7b53ac-67ad-4a8e-8691-8e85cbd05556 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:28.766518 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7b53ac-67ad-4a8e-8691-8e85cbd05556 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:28.766809 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7b53ac-67ad-4a8e-8691-8e85cbd05556 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:28.785108 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-0e7b53ac-67ad-4a8e-8691-8e85cbd05556 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:30:28.786354 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-0e7b53ac-67ad-4a8e-8691-8e85cbd05556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:30:28.787717 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ef29df-a604-4bf8-8db4-3a3056a1df62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:28.788422 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ef29df-a604-4bf8-8db4-3a3056a1df62 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:28.788695 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ef29df-a604-4bf8-8db4-3a3056a1df62 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:28.788979 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ef29df-a604-4bf8-8db4-3a3056a1df62 None admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:28.790562 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ef29df-a604-4bf8-8db4-3a3056a1df62 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:30:28.790877 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ef29df-a604-4bf8-8db4-3a3056a1df62 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:28.792251 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ef29df-a604-4bf8-8db4-3a3056a1df62 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:28.801727 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 40/74] 10.222.0.89 () {56 vars in 1099 bytes} [Thu Jul 28 17:30:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:30:28.802633 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ef29df-a604-4bf8-8db4-3a3056a1df62 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:28.833977 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:28.845363 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:28.876715 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 35/75] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:30:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:30:28.878582 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eac5d2-9ce4-42bd-95d1-ef1b630ea32e None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:28.878877 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eac5d2-9ce4-42bd-95d1-ef1b630ea32e None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:28.879158 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eac5d2-9ce4-42bd-95d1-ef1b630ea32e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:28.901282 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-a5eac5d2-9ce4-42bd-95d1-ef1b630ea32e None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:30:28.902646 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-a5eac5d2-9ce4-42bd-95d1-ef1b630ea32e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:30:28.935209 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ef29df-a604-4bf8-8db4-3a3056a1df62 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:28.975245 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cad594-ad22-4088-92cd-208c9e78563e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:28.976221 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cad594-ad22-4088-92cd-208c9e78563e None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:28.976506 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cad594-ad22-4088-92cd-208c9e78563e None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:28.976785 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cad594-ad22-4088-92cd-208c9e78563e None admin] PATH_INFO: `/v3/users/7eafa4211ada43a0b09036090ff902f7` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:28.977870 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 36/76] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:30:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:30:28.978785 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5eac5d2-9ce4-42bd-95d1-ef1b630ea32e None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:29.000855 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cad594-ad22-4088-92cd-208c9e78563e None admin] RBAC: Authorizing `identity:get_user(user_id=7eafa4211ada43a0b09036090ff902f7)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:29.002410 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cad594-ad22-4088-92cd-208c9e78563e None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:29.004153 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 41/77] 10.222.0.89 () {56 vars in 1177 bytes} [Thu Jul 28 17:30:28 2022] GET /identity/v3/users/7eafa4211ada43a0b09036090ff902f7 => generated 275 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:30:29.004937 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cad594-ad22-4088-92cd-208c9e78563e None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:29.102430 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5eac5d2-9ce4-42bd-95d1-ef1b630ea32e None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:29.105051 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:29.109041 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:29.125770 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cad594-ad22-4088-92cd-208c9e78563e None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:29.144874 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a612bb59-4cc5-4b18-a20d-444b188a47e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:29.145482 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a612bb59-4cc5-4b18-a20d-444b188a47e5 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:29.146029 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a612bb59-4cc5-4b18-a20d-444b188a47e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:29.146314 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a612bb59-4cc5-4b18-a20d-444b188a47e5 None admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:29.147054 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a612bb59-4cc5-4b18-a20d-444b188a47e5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:30:29.147354 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a612bb59-4cc5-4b18-a20d-444b188a47e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:29.148777 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a612bb59-4cc5-4b18-a20d-444b188a47e5 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:29.162066 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 37/78] 10.222.0.89 () {56 vars in 1111 bytes} [Thu Jul 28 17:30:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:30:29.163410 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a612bb59-4cc5-4b18-a20d-444b188a47e5 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:29.174671 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f799f8-3721-4648-83ad-f9cd3ca8ece2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:29.175469 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f799f8-3721-4648-83ad-f9cd3ca8ece2 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:29.175875 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f799f8-3721-4648-83ad-f9cd3ca8ece2 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:29.176329 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f799f8-3721-4648-83ad-f9cd3ca8ece2 None admin] PATH_INFO: `/v3/users/bbc7ab320041413396df1eef4db728ae` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:29.178274 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f799f8-3721-4648-83ad-f9cd3ca8ece2 None admin] RBAC: Authorizing `identity:get_user(user_id=bbc7ab320041413396df1eef4db728ae)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:29.180556 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f799f8-3721-4648-83ad-f9cd3ca8ece2 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:29.186950 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 42/79] 10.222.0.89 () {56 vars in 1177 bytes} [Thu Jul 28 17:30:29 2022] GET /identity/v3/users/bbc7ab320041413396df1eef4db728ae => generated 247 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:30:29.186950 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f799f8-3721-4648-83ad-f9cd3ca8ece2 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:29.283804 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a612bb59-4cc5-4b18-a20d-444b188a47e5 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:29.314104 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f799f8-3721-4648-83ad-f9cd3ca8ece2 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:29.321059 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2dbddb-d120-4510-9078-ef831ad8c6ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:29.321738 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2dbddb-d120-4510-9078-ef831ad8c6ff None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:29.322019 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2dbddb-d120-4510-9078-ef831ad8c6ff None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:29.322295 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2dbddb-d120-4510-9078-ef831ad8c6ff None admin] PATH_INFO: `/v3/domains/default` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:29.323503 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2dbddb-d120-4510-9078-ef831ad8c6ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:29.324960 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2dbddb-d120-4510-9078-ef831ad8c6ff None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:29.331665 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 38/80] 10.222.0.89 () {56 vars in 1108 bytes} [Thu Jul 28 17:30:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:30:29.331665 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2dbddb-d120-4510-9078-ef831ad8c6ff None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:29.357738 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3928b968-b5a3-40bd-8d29-311507532f63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:29.358491 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3928b968-b5a3-40bd-8d29-311507532f63 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:29.359578 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3928b968-b5a3-40bd-8d29-311507532f63 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:29.359578 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3928b968-b5a3-40bd-8d29-311507532f63 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:29.370650 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:29.372705 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3928b968-b5a3-40bd-8d29-311507532f63 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:30:29.378564 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-3928b968-b5a3-40bd-8d29-311507532f63 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 17:30:29.378564 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:30:29.378564 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:30:29.378564 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:30:29.378564 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:30:29.378564 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:30:29.378564 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:30:29.378564 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:30:29.378564 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:30:29.378564 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:30:29.378564 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:30:29.378564 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:30:29.378564 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:30:29.378564 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:30:29.378564 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:30:29.378564 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:30:29.379452 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:30:29.379452 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:30:29.379452 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:30:29.379452 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:30:29.379452 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:30:29.379452 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:30:29.379452 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:30:29.379452 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:30:29.379452 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:30:29.379452 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:30:29.379452 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:30:29.379452 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:30:29.379452 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:30:29.379452 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:30:29.379452 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:30:29.379452 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:30:29.380388 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:30:29.380388 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:30:29.380388 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:30:29.380388 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:30:29.380388 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:30:29.380388 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:30:29.380388 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:30:29.380388 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:30:29.380388 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:30:29.380388 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:30:29.380388 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 17:30:29.380388 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:30:29.380388 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 43/81] 10.222.0.89 () {56 vars in 1099 bytes} [Thu Jul 28 17:30:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:30:29.381272 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3928b968-b5a3-40bd-8d29-311507532f63 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:29.397891 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:29.450128 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2dbddb-d120-4510-9078-ef831ad8c6ff None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:29.486682 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fba414-e6da-4eea-9ddb-66342e6d145e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:29.487310 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fba414-e6da-4eea-9ddb-66342e6d145e None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:29.487594 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fba414-e6da-4eea-9ddb-66342e6d145e None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:29.487904 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fba414-e6da-4eea-9ddb-66342e6d145e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:29.497504 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3928b968-b5a3-40bd-8d29-311507532f63 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:29.538713 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e3e65d-4378-446a-bf11-09506a01966e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:29.539371 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e3e65d-4378-446a-bf11-09506a01966e None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:29.540022 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e3e65d-4378-446a-bf11-09506a01966e None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:29.540298 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e3e65d-4378-446a-bf11-09506a01966e None admin] PATH_INFO: `/v3/users/7eafa4211ada43a0b09036090ff902f7` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:29.541804 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e3e65d-4378-446a-bf11-09506a01966e None admin] RBAC: Authorizing `identity:get_user(user_id=7eafa4211ada43a0b09036090ff902f7)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:29.543421 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e3e65d-4378-446a-bf11-09506a01966e None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:29.545032 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 44/82] 10.222.0.89 () {56 vars in 1177 bytes} [Thu Jul 28 17:30:29 2022] GET /identity/v3/users/7eafa4211ada43a0b09036090ff902f7 => generated 275 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:30:29.545880 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e3e65d-4378-446a-bf11-09506a01966e None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:29.564949 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fba414-e6da-4eea-9ddb-66342e6d145e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:29.566617 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fba414-e6da-4eea-9ddb-66342e6d145e None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:29.605289 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:29.641579 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:29.667236 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e3e65d-4378-446a-bf11-09506a01966e None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:29.680533 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 39/83] 10.222.0.89 () {58 vars in 1282 bytes} [Thu Jul 28 17:30:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 17:30:29.682300 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fba414-e6da-4eea-9ddb-66342e6d145e None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:29.706606 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56a048f-7f65-4e56-ab35-c52e97770d47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:29.707234 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56a048f-7f65-4e56-ab35-c52e97770d47 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:29.707517 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56a048f-7f65-4e56-ab35-c52e97770d47 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:29.707790 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56a048f-7f65-4e56-ab35-c52e97770d47 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:29.785139 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56a048f-7f65-4e56-ab35-c52e97770d47 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:29.786643 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56a048f-7f65-4e56-ab35-c52e97770d47 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:29.806311 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fba414-e6da-4eea-9ddb-66342e6d145e None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:29.849489 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce84be9-2207-4974-923a-3e775bf860ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:29.850143 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce84be9-2207-4974-923a-3e775bf860ce None admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:29.850427 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce84be9-2207-4974-923a-3e775bf860ce None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:29.850792 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce84be9-2207-4974-923a-3e775bf860ce None admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:29.851405 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce84be9-2207-4974-923a-3e775bf860ce None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:29.852861 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce84be9-2207-4974-923a-3e775bf860ce None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:29.861473 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:29.877658 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 40/84] 10.222.0.89 () {60 vars in 1132 bytes} [Thu Jul 28 17:30:29 2022] POST /identity/v3/users => generated 289 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:30:29.878637 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce84be9-2207-4974-923a-3e775bf860ce None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:29.886553 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:29.912229 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 45/85] 10.222.0.89 () {58 vars in 1282 bytes} [Thu Jul 28 17:30:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 17:30:29.913509 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56a048f-7f65-4e56-ab35-c52e97770d47 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:30.022349 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce84be9-2207-4974-923a-3e775bf860ce None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:30.050106 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56a048f-7f65-4e56-ab35-c52e97770d47 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:30.069283 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2300b5-f7d8-4db1-aede-75eb61c004c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:30.070301 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2300b5-f7d8-4db1-aede-75eb61c004c5 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:30.070742 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2300b5-f7d8-4db1-aede-75eb61c004c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:30.071162 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2300b5-f7d8-4db1-aede-75eb61c004c5 None admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:30.072841 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2300b5-f7d8-4db1-aede-75eb61c004c5 None admin] RBAC: Adding query filter params (name=member) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:30:30.074214 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2300b5-f7d8-4db1-aede-75eb61c004c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:30.076055 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2300b5-f7d8-4db1-aede-75eb61c004c5 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:30.092787 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 41/86] 10.222.0.89 () {56 vars in 1101 bytes} [Thu Jul 28 17:30:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:30:30.093748 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2300b5-f7d8-4db1-aede-75eb61c004c5 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:30.107334 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecfc5f0-d402-4d62-8a5f-b3140989eddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:30.107334 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecfc5f0-d402-4d62-8a5f-b3140989eddc None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:30.107334 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecfc5f0-d402-4d62-8a5f-b3140989eddc None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:30.107671 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecfc5f0-d402-4d62-8a5f-b3140989eddc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:30.183855 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:30.198440 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:30.225989 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecfc5f0-d402-4d62-8a5f-b3140989eddc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:30.229075 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecfc5f0-d402-4d62-8a5f-b3140989eddc None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:30.263459 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2300b5-f7d8-4db1-aede-75eb61c004c5 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:30.361270 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2076af-6313-4f6a-9e67-8df0b3dbe621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:30.361270 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2076af-6313-4f6a-9e67-8df0b3dbe621 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:30.361270 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2076af-6313-4f6a-9e67-8df0b3dbe621 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:30.361270 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2076af-6313-4f6a-9e67-8df0b3dbe621 None admin] PATH_INFO: `/v3/projects/26339f3896c14d148dd3af450ec9d25f` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:30.361638 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2076af-6313-4f6a-9e67-8df0b3dbe621 None admin] RBAC: Authorizing `identity:get_project(project_id=26339f3896c14d148dd3af450ec9d25f)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:30.361899 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2076af-6313-4f6a-9e67-8df0b3dbe621 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:30.370417 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 42/87] 10.222.0.89 () {56 vars in 1186 bytes} [Thu Jul 28 17:30:30 2022] GET /identity/v3/projects/26339f3896c14d148dd3af450ec9d25f => generated 298 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:30:30.371994 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2076af-6313-4f6a-9e67-8df0b3dbe621 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:30.430948 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 46/88] 10.222.0.89 () {58 vars in 1282 bytes} [Thu Jul 28 17:30:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 516 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 17:30:30.430948 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecfc5f0-d402-4d62-8a5f-b3140989eddc None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:30.540042 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2076af-6313-4f6a-9e67-8df0b3dbe621 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:30.555579 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:30.581892 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:30.589182 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecfc5f0-d402-4d62-8a5f-b3140989eddc None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:30.617262 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae385e5a-f2f5-463b-91cc-ba7cb96ec925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:30.617262 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae385e5a-f2f5-463b-91cc-ba7cb96ec925 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:30.617262 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae385e5a-f2f5-463b-91cc-ba7cb96ec925 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:30.617262 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae385e5a-f2f5-463b-91cc-ba7cb96ec925 None admin] PATH_INFO: `/v3/projects/26339f3896c14d148dd3af450ec9d25f` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:30.617670 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae385e5a-f2f5-463b-91cc-ba7cb96ec925 None admin] RBAC: Authorizing `identity:get_project(project_id=26339f3896c14d148dd3af450ec9d25f)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:30.618502 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae385e5a-f2f5-463b-91cc-ba7cb96ec925 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:30.625181 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 43/89] 10.222.0.89 () {56 vars in 1186 bytes} [Thu Jul 28 17:30:30 2022] GET /identity/v3/projects/26339f3896c14d148dd3af450ec9d25f => generated 298 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:30:30.625181 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae385e5a-f2f5-463b-91cc-ba7cb96ec925 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:30.675696 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f74094-06e3-4efb-af76-8f07db32842a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:30.676358 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f74094-06e3-4efb-af76-8f07db32842a None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:30.676648 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f74094-06e3-4efb-af76-8f07db32842a None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:30.676928 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f74094-06e3-4efb-af76-8f07db32842a None admin] PATH_INFO: `/v3/users/7eafa4211ada43a0b09036090ff902f7` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:30.681959 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f74094-06e3-4efb-af76-8f07db32842a None admin] RBAC: Authorizing `identity:get_user(user_id=7eafa4211ada43a0b09036090ff902f7)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:30.683477 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f74094-06e3-4efb-af76-8f07db32842a None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:30.685128 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 47/90] 10.222.0.89 () {56 vars in 1177 bytes} [Thu Jul 28 17:30:30 2022] GET /identity/v3/users/7eafa4211ada43a0b09036090ff902f7 => generated 275 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:30:30.691776 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f74094-06e3-4efb-af76-8f07db32842a None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:30.826614 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae385e5a-f2f5-463b-91cc-ba7cb96ec925 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:30.900101 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7861fc04-fa19-495b-b49d-5ff575e92830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:30.901424 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7861fc04-fa19-495b-b49d-5ff575e92830 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:30.901645 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7861fc04-fa19-495b-b49d-5ff575e92830 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:30.901922 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7861fc04-fa19-495b-b49d-5ff575e92830 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:30.902712 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7861fc04-fa19-495b-b49d-5ff575e92830 None admin] RBAC: Adding query filter params (role.id=11f8731756fe4caca13efa61b71b11c5, scope.project.id=26339f3896c14d148dd3af450ec9d25f, user.id=7eafa4211ada43a0b09036090ff902f7) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:30:30.903022 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7861fc04-fa19-495b-b49d-5ff575e92830 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:30.904467 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7861fc04-fa19-495b-b49d-5ff575e92830 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:30.910125 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f74094-06e3-4efb-af76-8f07db32842a None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:30.925015 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 44/91] 10.222.0.89 () {56 vars in 1374 bytes} [Thu Jul 28 17:30:30 2022] GET /identity/v3/role_assignments?user.id=7eafa4211ada43a0b09036090ff902f7&scope.project.id=26339f3896c14d148dd3af450ec9d25f&role.id=11f8731756fe4caca13efa61b71b11c5 => generated 259 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:30:30.925015 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7861fc04-fa19-495b-b49d-5ff575e92830 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:30.979606 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af65190-6719-4000-8bdc-f7215bd1aebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:30.980520 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af65190-6719-4000-8bdc-f7215bd1aebb None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:30.980954 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af65190-6719-4000-8bdc-f7215bd1aebb None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:30.981393 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af65190-6719-4000-8bdc-f7215bd1aebb None admin] PATH_INFO: `/v3/projects/26339f3896c14d148dd3af450ec9d25f` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:30.985912 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af65190-6719-4000-8bdc-f7215bd1aebb None admin] RBAC: Authorizing `identity:get_project(project_id=26339f3896c14d148dd3af450ec9d25f)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:30.987901 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af65190-6719-4000-8bdc-f7215bd1aebb None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:30.989901 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 48/92] 10.222.0.89 () {56 vars in 1186 bytes} [Thu Jul 28 17:30:30 2022] GET /identity/v3/projects/26339f3896c14d148dd3af450ec9d25f => generated 298 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:30:30.998326 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af65190-6719-4000-8bdc-f7215bd1aebb None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:31.041745 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:31.047460 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:31.125760 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7861fc04-fa19-495b-b49d-5ff575e92830 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:31.170484 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af65190-6719-4000-8bdc-f7215bd1aebb None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:31.218787 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd44dfb9-ae57-4d50-a321-2e4970cc2283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:31.219020 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe08ab4-82ef-4768-8a94-7d855cac7856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:31.219893 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd44dfb9-ae57-4d50-a321-2e4970cc2283 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:31.220186 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe08ab4-82ef-4768-8a94-7d855cac7856 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:31.220471 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe08ab4-82ef-4768-8a94-7d855cac7856 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:31.220745 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe08ab4-82ef-4768-8a94-7d855cac7856 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:31.221797 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd44dfb9-ae57-4d50-a321-2e4970cc2283 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:31.222307 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd44dfb9-ae57-4d50-a321-2e4970cc2283 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:31.223704 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd44dfb9-ae57-4d50-a321-2e4970cc2283 None admin] RBAC: Adding query filter params (role.id=73e457231a824ea98c28d22610be640c, scope.project.id=26339f3896c14d148dd3af450ec9d25f, user.id=bbc7ab320041413396df1eef4db728ae) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:30:31.224173 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd44dfb9-ae57-4d50-a321-2e4970cc2283 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:31.236323 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd44dfb9-ae57-4d50-a321-2e4970cc2283 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:31.249222 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 45/93] 10.222.0.89 () {56 vars in 1374 bytes} [Thu Jul 28 17:30:30 2022] GET /identity/v3/role_assignments?user.id=bbc7ab320041413396df1eef4db728ae&scope.project.id=26339f3896c14d148dd3af450ec9d25f&role.id=73e457231a824ea98c28d22610be640c => generated 259 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:30:31.249222 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd44dfb9-ae57-4d50-a321-2e4970cc2283 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:31.470308 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe08ab4-82ef-4768-8a94-7d855cac7856 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:31.471793 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe08ab4-82ef-4768-8a94-7d855cac7856 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:31.528931 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:31.587235 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd44dfb9-ae57-4d50-a321-2e4970cc2283 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:31.614514 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:31.673185 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ec0d6a-2b2e-4430-abef-64f37b1fce5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:31.673185 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ec0d6a-2b2e-4430-abef-64f37b1fce5c None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:31.673185 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ec0d6a-2b2e-4430-abef-64f37b1fce5c None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:31.673185 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ec0d6a-2b2e-4430-abef-64f37b1fce5c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:31.673597 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ec0d6a-2b2e-4430-abef-64f37b1fce5c None admin] RBAC: Adding query filter params (role.id=895fd016dffb41e7b0205c9a55a4e0f8, scope.project.id=26339f3896c14d148dd3af450ec9d25f, user.id=7eafa4211ada43a0b09036090ff902f7) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:30:31.673597 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ec0d6a-2b2e-4430-abef-64f37b1fce5c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:31.674078 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ec0d6a-2b2e-4430-abef-64f37b1fce5c None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:31.680351 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 46/94] 10.222.0.89 () {56 vars in 1374 bytes} [Thu Jul 28 17:30:31 2022] GET /identity/v3/role_assignments?user.id=7eafa4211ada43a0b09036090ff902f7&scope.project.id=26339f3896c14d148dd3af450ec9d25f&role.id=895fd016dffb41e7b0205c9a55a4e0f8 => generated 259 bytes in 434 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:30:31.721045 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 49/95] 10.222.0.89 () {58 vars in 1282 bytes} [Thu Jul 28 17:30:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 723 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 17:30:31.727228 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ec0d6a-2b2e-4430-abef-64f37b1fce5c None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:31.910020 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ec0d6a-2b2e-4430-abef-64f37b1fce5c None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:31.973679 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e246670-fb69-4620-960d-0db7c6ccbed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:31.974196 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e246670-fb69-4620-960d-0db7c6ccbed4 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:31.974481 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e246670-fb69-4620-960d-0db7c6ccbed4 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:31.974754 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e246670-fb69-4620-960d-0db7c6ccbed4 None admin] PATH_INFO: `/v3/projects/38d2f8adcb494ba8b566e63d96beceda` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:31.980152 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e246670-fb69-4620-960d-0db7c6ccbed4 None admin] RBAC: Authorizing `identity:get_project(project_id=38d2f8adcb494ba8b566e63d96beceda)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:31.982773 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e246670-fb69-4620-960d-0db7c6ccbed4 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:31.983673 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 47/96] 10.222.0.89 () {56 vars in 1186 bytes} [Thu Jul 28 17:30:31 2022] GET /identity/v3/projects/38d2f8adcb494ba8b566e63d96beceda => generated 312 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:30:31.993063 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe08ab4-82ef-4768-8a94-7d855cac7856 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:32.165354 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe08ab4-82ef-4768-8a94-7d855cac7856 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:32.191902 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:32.227983 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f1b6f9-bf64-45e9-9e7e-360cd07fa05d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:32.228558 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f1b6f9-bf64-45e9-9e7e-360cd07fa05d None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:32.228845 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f1b6f9-bf64-45e9-9e7e-360cd07fa05d None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:32.229123 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f1b6f9-bf64-45e9-9e7e-360cd07fa05d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:32.230038 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f1b6f9-bf64-45e9-9e7e-360cd07fa05d None admin] RBAC: Adding query filter params (role.id=11f8731756fe4caca13efa61b71b11c5, scope.project.id=38d2f8adcb494ba8b566e63d96beceda, user.id=7eafa4211ada43a0b09036090ff902f7) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:30:32.233182 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f1b6f9-bf64-45e9-9e7e-360cd07fa05d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:32.233182 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f1b6f9-bf64-45e9-9e7e-360cd07fa05d None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:32.254909 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 50/97] 10.222.0.89 () {56 vars in 1374 bytes} [Thu Jul 28 17:30:31 2022] GET /identity/v3/role_assignments?user.id=7eafa4211ada43a0b09036090ff902f7&scope.project.id=38d2f8adcb494ba8b566e63d96beceda&role.id=11f8731756fe4caca13efa61b71b11c5 => generated 259 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:30:32.610917 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933b900f-e4ea-44ee-bf96-f8c1d3dcac0c None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:32.611477 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933b900f-e4ea-44ee-bf96-f8c1d3dcac0c None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:32.611477 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933b900f-e4ea-44ee-bf96-f8c1d3dcac0c None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:32.612843 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 48/98] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:30:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:30:32.619914 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d2b2d8-b3d0-47ea-a731-cf59b0a77b66 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:32.619914 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d2b2d8-b3d0-47ea-a731-cf59b0a77b66 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:32.619914 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d2b2d8-b3d0-47ea-a731-cf59b0a77b66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:32.654221 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-85d2b2d8-b3d0-47ea-a731-cf59b0a77b66 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:30:32.656710 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-85d2b2d8-b3d0-47ea-a731-cf59b0a77b66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:30:32.795686 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 51/99] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:30:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 17:30:32.850290 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933b900f-e4ea-44ee-bf96-f8c1d3dcac0c None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:32.897769 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d7a8f7-605a-46b9-ac84-469b66c1e833 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:32.898050 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d7a8f7-605a-46b9-ac84-469b66c1e833 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:32.899740 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d7a8f7-605a-46b9-ac84-469b66c1e833 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:32.899740 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 52/100] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:30:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 17:30:32.906150 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f0338b-4221-4b88-baa3-630192c17c75 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:32.906493 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f0338b-4221-4b88-baa3-630192c17c75 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:32.908354 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f0338b-4221-4b88-baa3-630192c17c75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:32.937528 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:32.946583 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-e4f0338b-4221-4b88-baa3-630192c17c75 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:30:32.947935 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-e4f0338b-4221-4b88-baa3-630192c17c75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:30:33.064200 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933b900f-e4ea-44ee-bf96-f8c1d3dcac0c None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:33.145267 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 53/101] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:30:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 17:30:33.184700 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f0338b-4221-4b88-baa3-630192c17c75 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:33.195837 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f4195f-1a3c-4697-a54f-01f9cc0fb6e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:33.196826 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f4195f-1a3c-4697-a54f-01f9cc0fb6e9 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:33.197342 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f4195f-1a3c-4697-a54f-01f9cc0fb6e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:33.197856 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f4195f-1a3c-4697-a54f-01f9cc0fb6e9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:33.226173 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f4195f-1a3c-4697-a54f-01f9cc0fb6e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:30:33.231640 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-a2f4195f-1a3c-4697-a54f-01f9cc0fb6e9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 17:30:33.231640 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:30:33.231640 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:30:33.231640 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:30:33.231640 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:30:33.231640 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:30:33.231640 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:30:33.231640 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:30:33.231640 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:30:33.231640 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:30:33.231640 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:30:33.231640 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:30:33.231640 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:30:33.231640 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:30:33.231640 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:30:33.231640 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:30:33.232475 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:30:33.232475 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:30:33.232475 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:30:33.232475 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:30:33.232475 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:30:33.232475 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:30:33.232475 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:30:33.232475 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:30:33.232475 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:30:33.232475 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:30:33.232475 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:30:33.232475 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( Jul 28 17:30:33.232475 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:30:33.232475 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:30:33.232475 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:30:33.232475 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:30:33.233338 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:30:33.233338 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:30:33.233338 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:30:33.233338 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:30:33.233338 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:30:33.233338 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:30:33.233338 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:30:33.233338 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:30:33.233338 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:30:33.233338 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:30:33.233338 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 17:30:33.233338 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:30:33.233338 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 49/102] 10.222.0.89 () {56 vars in 1099 bytes} [Thu Jul 28 17:30:32 2022] GET /identity/v3/roles/reader => generated 84 bytes in 383 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:30:33.240616 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f4195f-1a3c-4697-a54f-01f9cc0fb6e9 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:33.280726 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:33.448262 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f0338b-4221-4b88-baa3-630192c17c75 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:33.539538 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e340405-15aa-4679-a337-0565d78eada6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:33.541111 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e340405-15aa-4679-a337-0565d78eada6 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:33.541421 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e340405-15aa-4679-a337-0565d78eada6 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:33.541685 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e340405-15aa-4679-a337-0565d78eada6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:33.546762 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e340405-15aa-4679-a337-0565d78eada6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:33.548543 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e340405-15aa-4679-a337-0565d78eada6 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:33.550270 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 54/103] 10.222.0.89 () {56 vars in 1108 bytes} [Thu Jul 28 17:30:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 366 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:30:33.558585 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e340405-15aa-4679-a337-0565d78eada6 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:33.636853 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f4195f-1a3c-4697-a54f-01f9cc0fb6e9 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:33.653101 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:33.707537 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb92e93-52d6-4b27-8254-d601b6f5dff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:33.708096 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb92e93-52d6-4b27-8254-d601b6f5dff5 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:33.708377 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb92e93-52d6-4b27-8254-d601b6f5dff5 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:33.708657 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb92e93-52d6-4b27-8254-d601b6f5dff5 None admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:33.709422 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb92e93-52d6-4b27-8254-d601b6f5dff5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:30:33.709728 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb92e93-52d6-4b27-8254-d601b6f5dff5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:33.711121 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb92e93-52d6-4b27-8254-d601b6f5dff5 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:33.729631 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 50/104] 10.222.0.89 () {56 vars in 1101 bytes} [Thu Jul 28 17:30:33 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 489 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:30:33.742258 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb92e93-52d6-4b27-8254-d601b6f5dff5 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:33.919203 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e340405-15aa-4679-a337-0565d78eada6 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:33.948295 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:33.981247 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb92e93-52d6-4b27-8254-d601b6f5dff5 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:34.020902 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c052be-8417-454f-8ceb-c6f5405e8ed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:34.021566 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c052be-8417-454f-8ceb-c6f5405e8ed0 None admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:34.021847 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c052be-8417-454f-8ceb-c6f5405e8ed0 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:34.022116 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c052be-8417-454f-8ceb-c6f5405e8ed0 None admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:34.022804 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c052be-8417-454f-8ceb-c6f5405e8ed0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:34.024241 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c052be-8417-454f-8ceb-c6f5405e8ed0 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:34.081681 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cce2c5-8282-478b-94ee-add2b4179755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:34.082336 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cce2c5-8282-478b-94ee-add2b4179755 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:34.082612 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cce2c5-8282-478b-94ee-add2b4179755 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:34.082889 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cce2c5-8282-478b-94ee-add2b4179755 None admin] PATH_INFO: `/v3/users/5c88f62bf45d45afade134bc5ff1e000` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:34.104403 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 55/105] 10.222.0.89 () {60 vars in 1141 bytes} [Thu Jul 28 17:30:33 2022] POST /identity/v3/projects => generated 302 bytes in 546 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:30:34.122087 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cce2c5-8282-478b-94ee-add2b4179755 None admin] RBAC: Authorizing `identity:get_user(user_id=5c88f62bf45d45afade134bc5ff1e000)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:34.123685 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cce2c5-8282-478b-94ee-add2b4179755 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:34.126628 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 51/106] 10.222.0.89 () {56 vars in 1177 bytes} [Thu Jul 28 17:30:33 2022] GET /identity/v3/users/5c88f62bf45d45afade134bc5ff1e000 => generated 289 bytes in 390 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:30:34.132662 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c052be-8417-454f-8ceb-c6f5405e8ed0 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:34.296574 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451528ec-f914-40cd-b577-9b8814a19e55 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:34.296913 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451528ec-f914-40cd-b577-9b8814a19e55 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:34.297283 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451528ec-f914-40cd-b577-9b8814a19e55 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:34.298822 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 52/107] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:30:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:30:34.314169 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c53faf-c217-443c-984f-3b09b9328b96 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:34.314473 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c53faf-c217-443c-984f-3b09b9328b96 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:34.314749 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c53faf-c217-443c-984f-3b09b9328b96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:34.367715 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-e4c53faf-c217-443c-984f-3b09b9328b96 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:30:34.381891 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-e4c53faf-c217-443c-984f-3b09b9328b96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:30:34.414251 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c052be-8417-454f-8ceb-c6f5405e8ed0 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:34.436214 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:34.481588 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1ae57f-9729-4b5d-977d-c009791ea52d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:34.482308 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1ae57f-9729-4b5d-977d-c009791ea52d None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:34.482876 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1ae57f-9729-4b5d-977d-c009791ea52d None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:34.482876 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1ae57f-9729-4b5d-977d-c009791ea52d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:34.537255 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 53/108] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:30:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 17:30:34.553957 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:34.579142 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c53faf-c217-443c-984f-3b09b9328b96 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:34.614854 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1ae57f-9729-4b5d-977d-c009791ea52d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:34.616745 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1ae57f-9729-4b5d-977d-c009791ea52d None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:34.785989 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c53faf-c217-443c-984f-3b09b9328b96 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:34.844813 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fbc6d7-14dc-4c6e-be54-01643a5e63ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:34.845559 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fbc6d7-14dc-4c6e-be54-01643a5e63ff None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:34.845844 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fbc6d7-14dc-4c6e-be54-01643a5e63ff None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:34.846123 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fbc6d7-14dc-4c6e-be54-01643a5e63ff None admin] PATH_INFO: `/v3/roles/member` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:34.888609 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fbc6d7-14dc-4c6e-be54-01643a5e63ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:30:34.894550 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-11fbc6d7-14dc-4c6e-be54-01643a5e63ff None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 17:30:34.894550 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:30:34.894550 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:30:34.894550 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:30:34.894550 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:30:34.894550 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:30:34.894550 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:30:34.894550 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:30:34.894550 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:30:34.894550 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:30:34.894550 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:30:34.894550 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:30:34.894550 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:30:34.894550 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:30:34.894550 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:30:34.894550 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:30:34.895406 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:30:34.895406 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:30:34.895406 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:30:34.895406 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:30:34.895406 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:30:34.895406 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:30:34.895406 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:30:34.895406 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:30:34.895406 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:30:34.895406 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:30:34.895406 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:30:34.895406 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( Jul 28 17:30:34.895406 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:30:34.895406 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:30:34.895406 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:30:34.895406 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:30:34.896171 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:30:34.896171 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:30:34.896171 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:30:34.896171 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:30:34.896171 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:30:34.896171 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:30:34.896171 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:30:34.896171 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:30:34.896171 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:30:34.896171 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:30:34.896171 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 17:30:34.896171 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:30:34.896171 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 54/109] 10.222.0.89 () {56 vars in 1099 bytes} [Thu Jul 28 17:30:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:30:34.903198 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 56/110] 10.222.0.89 () {58 vars in 1282 bytes} [Thu Jul 28 17:30:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 771 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 17:30:34.904936 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cca03cf-4fd1-4c0d-8366-755f4a7843c3 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:34.906335 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1ae57f-9729-4b5d-977d-c009791ea52d None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:34.907004 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cca03cf-4fd1-4c0d-8366-755f4a7843c3 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:34.907297 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cca03cf-4fd1-4c0d-8366-755f4a7843c3 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:34.909918 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 55/111] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:30:34 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:30:34.911820 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cca03cf-4fd1-4c0d-8366-755f4a7843c3 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:34.959161 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:35.032524 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:35.122168 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1ae57f-9729-4b5d-977d-c009791ea52d None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:35.151432 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cca03cf-4fd1-4c0d-8366-755f4a7843c3 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:35.199821 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824e5688-89da-4000-b1cc-f498421621dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:35.201537 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824e5688-89da-4000-b1cc-f498421621dc None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:35.203766 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824e5688-89da-4000-b1cc-f498421621dc None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:35.204223 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824e5688-89da-4000-b1cc-f498421621dc None admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:35.206071 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824e5688-89da-4000-b1cc-f498421621dc None admin] RBAC: Adding query filter params (name=member) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:30:35.206951 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824e5688-89da-4000-b1cc-f498421621dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:35.209403 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824e5688-89da-4000-b1cc-f498421621dc None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:35.219851 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1014d402-04e3-41c1-aec0-45b421908a80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:35.220780 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1014d402-04e3-41c1-aec0-45b421908a80 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:35.221228 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1014d402-04e3-41c1-aec0-45b421908a80 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:35.221644 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1014d402-04e3-41c1-aec0-45b421908a80 None admin] PATH_INFO: `/v3/projects/26339f3896c14d148dd3af450ec9d25f` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:35.224058 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1014d402-04e3-41c1-aec0-45b421908a80 None admin] RBAC: Authorizing `identity:get_project(project_id=26339f3896c14d148dd3af450ec9d25f)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:35.228518 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1014d402-04e3-41c1-aec0-45b421908a80 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:35.230820 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 56/112] 10.222.0.89 () {56 vars in 1186 bytes} [Thu Jul 28 17:30:34 2022] GET /identity/v3/projects/26339f3896c14d148dd3af450ec9d25f => generated 298 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:30:35.237151 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 57/113] 10.222.0.89 () {56 vars in 1101 bytes} [Thu Jul 28 17:30:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:30:35.238792 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a350ff-e810-46f7-b5ae-4f26664b0a91 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:35.240132 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a350ff-e810-46f7-b5ae-4f26664b0a91 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:35.240559 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a350ff-e810-46f7-b5ae-4f26664b0a91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:35.244349 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b7499b-5228-4b6a-90e8-c2e3e65ef68e None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:35.244820 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b7499b-5228-4b6a-90e8-c2e3e65ef68e None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:35.245266 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b7499b-5228-4b6a-90e8-c2e3e65ef68e None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:35.246759 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 57/114] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:30:35 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:30:35.247613 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b7499b-5228-4b6a-90e8-c2e3e65ef68e None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:35.284827 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-62a350ff-e810-46f7-b5ae-4f26664b0a91 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:30:35.287760 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-62a350ff-e810-46f7-b5ae-4f26664b0a91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:30:35.383696 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:35.395305 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 58/115] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:30:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:30:35.395305 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a350ff-e810-46f7-b5ae-4f26664b0a91 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:35.409890 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b7499b-5228-4b6a-90e8-c2e3e65ef68e None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:35.454912 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fd8921-13bb-4253-ac7a-f83c413fe910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:35.455865 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fd8921-13bb-4253-ac7a-f83c413fe910 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:35.456256 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fd8921-13bb-4253-ac7a-f83c413fe910 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:35.456663 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fd8921-13bb-4253-ac7a-f83c413fe910 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:35.457653 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fd8921-13bb-4253-ac7a-f83c413fe910 None admin] RBAC: Adding query filter params (role.id=466df921c2db4456b37637c23382d92a, scope.project.id=26339f3896c14d148dd3af450ec9d25f, user.id=5c88f62bf45d45afade134bc5ff1e000) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:30:35.458124 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fd8921-13bb-4253-ac7a-f83c413fe910 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:35.459720 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fd8921-13bb-4253-ac7a-f83c413fe910 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:35.465572 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 58/116] 10.222.0.89 () {56 vars in 1374 bytes} [Thu Jul 28 17:30:35 2022] GET /identity/v3/role_assignments?user.id=5c88f62bf45d45afade134bc5ff1e000&scope.project.id=26339f3896c14d148dd3af450ec9d25f&role.id=466df921c2db4456b37637c23382d92a => generated 259 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:30:35.466823 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccc46da-20a9-4285-990c-1adb9994fd21 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:35.467179 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccc46da-20a9-4285-990c-1adb9994fd21 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:35.467525 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccc46da-20a9-4285-990c-1adb9994fd21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:35.473618 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:35.489479 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-8ccc46da-20a9-4285-990c-1adb9994fd21 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:30:35.491219 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-8ccc46da-20a9-4285-990c-1adb9994fd21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:30:35.582465 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a350ff-e810-46f7-b5ae-4f26664b0a91 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:35.587849 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 59/117] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:30:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:30:35.589489 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f32b347-e280-45e9-a551-89cdfe1fc9a3 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:35.589782 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f32b347-e280-45e9-a551-89cdfe1fc9a3 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:35.590056 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f32b347-e280-45e9-a551-89cdfe1fc9a3 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:35.591453 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 60/118] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:30:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:30:35.592448 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f32b347-e280-45e9-a551-89cdfe1fc9a3 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:35.648458 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dbacf0-d18b-4dcf-ab81-c2d403ed6085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:35.653187 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dbacf0-d18b-4dcf-ab81-c2d403ed6085 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:35.653187 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dbacf0-d18b-4dcf-ab81-c2d403ed6085 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:35.653187 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dbacf0-d18b-4dcf-ab81-c2d403ed6085 None admin] PATH_INFO: `/v3/users/7eafa4211ada43a0b09036090ff902f7` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:35.653187 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1dbacf0-d18b-4dcf-ab81-c2d403ed6085 None admin] RBAC: Authorizing `identity:get_user(user_id=7eafa4211ada43a0b09036090ff902f7)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:35.653786 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1dbacf0-d18b-4dcf-ab81-c2d403ed6085 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:35.661691 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 59/119] 10.222.0.89 () {56 vars in 1177 bytes} [Thu Jul 28 17:30:35 2022] GET /identity/v3/users/7eafa4211ada43a0b09036090ff902f7 => generated 275 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:30:35.663098 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea87768-1e24-4535-bfce-6ee2aa48d805 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:35.663384 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea87768-1e24-4535-bfce-6ee2aa48d805 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:35.669183 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea87768-1e24-4535-bfce-6ee2aa48d805 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:35.698695 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-6ea87768-1e24-4535-bfce-6ee2aa48d805 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:30:35.700200 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-6ea87768-1e24-4535-bfce-6ee2aa48d805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:30:35.729193 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:35.758554 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f32b347-e280-45e9-a551-89cdfe1fc9a3 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:35.805458 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 60/120] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:30:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:30:35.805458 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea87768-1e24-4535-bfce-6ee2aa48d805 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:35.862541 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa3aad2-f3ae-4019-ba34-01da6acb09a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:35.863376 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa3aad2-f3ae-4019-ba34-01da6acb09a4 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:35.863919 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa3aad2-f3ae-4019-ba34-01da6acb09a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:35.864409 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa3aad2-f3ae-4019-ba34-01da6acb09a4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:35.876020 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:35.887700 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa3aad2-f3ae-4019-ba34-01da6acb09a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:30:35.893846 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-aaa3aad2-f3ae-4019-ba34-01da6acb09a4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:30:35.893846 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:30:35.893846 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:30:35.893846 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:30:35.893846 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:30:35.893846 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:30:35.893846 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:30:35.893846 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:30:35.893846 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:30:35.893846 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:30:35.893846 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:30:35.893846 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:30:35.893846 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:30:35.893846 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:30:35.893846 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:30:35.893846 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:30:35.894723 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:30:35.894723 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:30:35.894723 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:30:35.894723 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:30:35.894723 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:30:35.894723 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:30:35.894723 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:30:35.894723 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:30:35.894723 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:30:35.894723 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:30:35.894723 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:30:35.894723 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( Jul 28 17:30:35.894723 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:30:35.894723 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:30:35.894723 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:30:35.894723 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:30:35.895467 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:30:35.895467 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:30:35.895467 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:30:35.895467 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:30:35.895467 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:30:35.895467 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:30:35.895467 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:30:35.895467 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:30:35.895467 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:30:35.895467 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:30:35.895467 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:30:35.895467 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:30:35.895467 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 61/121] 10.222.0.89 () {56 vars in 1096 bytes} [Thu Jul 28 17:30:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:30:35.896095 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa3aad2-f3ae-4019-ba34-01da6acb09a4 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:35.962150 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea87768-1e24-4535-bfce-6ee2aa48d805 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:36.012377 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02aed4d-1965-4f47-868b-247511f1875d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:36.013411 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02aed4d-1965-4f47-868b-247511f1875d None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:36.013886 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02aed4d-1965-4f47-868b-247511f1875d None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:36.014368 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02aed4d-1965-4f47-868b-247511f1875d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:36.028905 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02aed4d-1965-4f47-868b-247511f1875d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:30:36.035751 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-a02aed4d-1965-4f47-868b-247511f1875d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 17:30:36.035751 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:30:36.035751 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:30:36.035751 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:30:36.035751 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:30:36.035751 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:30:36.035751 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:30:36.035751 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:30:36.035751 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:30:36.035751 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:30:36.035751 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:30:36.035751 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:30:36.035751 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:30:36.035751 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:30:36.035751 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:30:36.035751 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:30:36.036613 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:30:36.036613 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:30:36.036613 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:30:36.036613 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:30:36.036613 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:30:36.036613 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:30:36.036613 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:30:36.036613 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:30:36.036613 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:30:36.036613 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:30:36.036613 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:30:36.036613 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:30:36.036613 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:30:36.036613 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:30:36.036613 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:30:36.036613 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:30:36.037401 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:30:36.037401 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:30:36.037401 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:30:36.037401 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:30:36.037401 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:30:36.037401 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:30:36.037401 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:30:36.037401 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:30:36.037401 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:30:36.037401 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:30:36.037401 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 17:30:36.037401 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:30:36.039722 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 61/122] 10.222.0.89 () {56 vars in 1114 bytes} [Thu Jul 28 17:30:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:30:36.042261 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02aed4d-1965-4f47-868b-247511f1875d None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:36.115904 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa3aad2-f3ae-4019-ba34-01da6acb09a4 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:36.178996 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57772cd-3595-46cb-b6f0-60ab12003a8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:36.179880 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57772cd-3595-46cb-b6f0-60ab12003a8f None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:36.180306 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57772cd-3595-46cb-b6f0-60ab12003a8f None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:36.180749 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57772cd-3595-46cb-b6f0-60ab12003a8f None admin] PATH_INFO: `/v3/projects/26339f3896c14d148dd3af450ec9d25f` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:36.182581 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57772cd-3595-46cb-b6f0-60ab12003a8f None admin] RBAC: Authorizing `identity:get_project(project_id=26339f3896c14d148dd3af450ec9d25f)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:36.184272 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57772cd-3595-46cb-b6f0-60ab12003a8f None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:36.188686 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 62/123] 10.222.0.89 () {56 vars in 1186 bytes} [Thu Jul 28 17:30:35 2022] GET /identity/v3/projects/26339f3896c14d148dd3af450ec9d25f => generated 298 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:30:36.190517 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57772cd-3595-46cb-b6f0-60ab12003a8f None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:36.196259 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02aed4d-1965-4f47-868b-247511f1875d None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:36.216319 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:36.216708 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:36.260992 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90938c05-3c49-4403-ac84-9aac0cb53cf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:36.261891 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90938c05-3c49-4403-ac84-9aac0cb53cf6 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:36.262269 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90938c05-3c49-4403-ac84-9aac0cb53cf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:36.262663 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90938c05-3c49-4403-ac84-9aac0cb53cf6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:36.282506 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90938c05-3c49-4403-ac84-9aac0cb53cf6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:30:36.294083 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-90938c05-3c49-4403-ac84-9aac0cb53cf6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 17:30:36.294083 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:30:36.294083 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:30:36.294083 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:30:36.294083 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:30:36.294083 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:30:36.294083 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:30:36.294083 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:30:36.294083 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:30:36.294083 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:30:36.294083 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:30:36.294083 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:30:36.294083 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:30:36.294083 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:30:36.294083 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:30:36.294083 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:30:36.295022 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:30:36.295022 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:30:36.295022 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:30:36.295022 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:30:36.295022 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:30:36.295022 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:30:36.295022 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:30:36.295022 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:30:36.295022 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:30:36.295022 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:30:36.295022 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:30:36.295022 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:30:36.295022 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:30:36.295022 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:30:36.295022 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:30:36.295022 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:30:36.295800 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:30:36.295800 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:30:36.295800 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:30:36.295800 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:30:36.295800 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:30:36.295800 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:30:36.295800 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:30:36.295800 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:30:36.295800 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:30:36.295800 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:30:36.295800 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 17:30:36.295800 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:30:36.309202 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 62/124] 10.222.0.89 () {56 vars in 1099 bytes} [Thu Jul 28 17:30:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:30:36.309202 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90938c05-3c49-4403-ac84-9aac0cb53cf6 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:36.362049 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57772cd-3595-46cb-b6f0-60ab12003a8f None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:36.409210 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ae7f94-3606-44f8-bb61-786adc7fdc8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:36.413178 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ae7f94-3606-44f8-bb61-786adc7fdc8b None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:36.413178 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ae7f94-3606-44f8-bb61-786adc7fdc8b None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:36.413178 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ae7f94-3606-44f8-bb61-786adc7fdc8b None admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:36.413178 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ae7f94-3606-44f8-bb61-786adc7fdc8b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:30:36.413178 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ae7f94-3606-44f8-bb61-786adc7fdc8b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:36.413178 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ae7f94-3606-44f8-bb61-786adc7fdc8b None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:36.439469 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 63/125] 10.222.0.89 () {56 vars in 1099 bytes} [Thu Jul 28 17:30:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:30:36.444082 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ae7f94-3606-44f8-bb61-786adc7fdc8b None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:36.525267 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90938c05-3c49-4403-ac84-9aac0cb53cf6 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:36.596001 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414659fb-886e-43ed-b2bc-3ecd748d7ea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:36.596634 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414659fb-886e-43ed-b2bc-3ecd748d7ea9 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:36.596913 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414659fb-886e-43ed-b2bc-3ecd748d7ea9 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:36.597206 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414659fb-886e-43ed-b2bc-3ecd748d7ea9 None admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:36.597935 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414659fb-886e-43ed-b2bc-3ecd748d7ea9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:30:36.598238 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414659fb-886e-43ed-b2bc-3ecd748d7ea9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:36.599621 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414659fb-886e-43ed-b2bc-3ecd748d7ea9 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:36.604545 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:36.626132 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 63/126] 10.222.0.89 () {56 vars in 1111 bytes} [Thu Jul 28 17:30:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:30:36.626132 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414659fb-886e-43ed-b2bc-3ecd748d7ea9 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:36.659826 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ae7f94-3606-44f8-bb61-786adc7fdc8b None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:36.707046 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:36.712617 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7214bd64-f782-4c4a-b807-fad5445f7c51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:36.715092 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7214bd64-f782-4c4a-b807-fad5445f7c51 None admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:36.715617 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7214bd64-f782-4c4a-b807-fad5445f7c51 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:36.716170 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7214bd64-f782-4c4a-b807-fad5445f7c51 None admin] PATH_INFO: `/v3/projects/26339f3896c14d148dd3af450ec9d25f/users/7eafa4211ada43a0b09036090ff902f7/roles/11f8731756fe4caca13efa61b71b11c5` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:36.718953 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7214bd64-f782-4c4a-b807-fad5445f7c51 None admin] RBAC: Authorizing `identity:create_grant(project_id=26339f3896c14d148dd3af450ec9d25f, user_id=7eafa4211ada43a0b09036090ff902f7, role_id=11f8731756fe4caca13efa61b71b11c5)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:36.720994 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7214bd64-f782-4c4a-b807-fad5445f7c51 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:36.742600 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 64/127] 10.222.0.89 () {58 vars in 1439 bytes} [Thu Jul 28 17:30:36 2022] PUT /identity/v3/projects/26339f3896c14d148dd3af450ec9d25f/users/7eafa4211ada43a0b09036090ff902f7/roles/11f8731756fe4caca13efa61b71b11c5 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:30:36.744271 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7214bd64-f782-4c4a-b807-fad5445f7c51 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:36.841185 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414659fb-886e-43ed-b2bc-3ecd748d7ea9 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:36.897185 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0ebe22-9e4c-449c-b8e1-f5320244de40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:36.897185 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0ebe22-9e4c-449c-b8e1-f5320244de40 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:36.897185 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0ebe22-9e4c-449c-b8e1-f5320244de40 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:36.897185 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0ebe22-9e4c-449c-b8e1-f5320244de40 None admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:36.897650 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0ebe22-9e4c-449c-b8e1-f5320244de40 None admin] RBAC: Adding query filter params (name=member) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:30:36.897650 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0ebe22-9e4c-449c-b8e1-f5320244de40 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:36.897650 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0ebe22-9e4c-449c-b8e1-f5320244de40 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:36.906127 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7214bd64-f782-4c4a-b807-fad5445f7c51 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:36.914726 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 64/128] 10.222.0.89 () {56 vars in 1101 bytes} [Thu Jul 28 17:30:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:30:36.915513 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0ebe22-9e4c-449c-b8e1-f5320244de40 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:36.958090 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:36.971825 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee08a4e-dae6-4aef-93c3-2c94b39310ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:36.972486 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee08a4e-dae6-4aef-93c3-2c94b39310ed None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:36.972769 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee08a4e-dae6-4aef-93c3-2c94b39310ed None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:36.973044 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee08a4e-dae6-4aef-93c3-2c94b39310ed None admin] PATH_INFO: `/v3/users/bbc7ab320041413396df1eef4db728ae` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:36.974924 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee08a4e-dae6-4aef-93c3-2c94b39310ed None admin] RBAC: Authorizing `identity:get_user(user_id=bbc7ab320041413396df1eef4db728ae)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:36.976387 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee08a4e-dae6-4aef-93c3-2c94b39310ed None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:36.978705 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 65/129] 10.222.0.89 () {56 vars in 1177 bytes} [Thu Jul 28 17:30:36 2022] GET /identity/v3/users/bbc7ab320041413396df1eef4db728ae => generated 247 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:30:36.979273 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437be155-1288-4f2e-9584-543e39db48eb None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:36.979579 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437be155-1288-4f2e-9584-543e39db48eb None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:36.979857 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437be155-1288-4f2e-9584-543e39db48eb None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:36.985338 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 66/130] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:30:36 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:30:36.986161 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437be155-1288-4f2e-9584-543e39db48eb None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:37.085645 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:37.096889 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0ebe22-9e4c-449c-b8e1-f5320244de40 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:37.122780 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437be155-1288-4f2e-9584-543e39db48eb None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:37.163311 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050ad536-3d84-4215-adc2-f1bdcf55c7c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:37.163990 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050ad536-3d84-4215-adc2-f1bdcf55c7c2 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:37.164256 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050ad536-3d84-4215-adc2-f1bdcf55c7c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:37.164531 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050ad536-3d84-4215-adc2-f1bdcf55c7c2 None admin] PATH_INFO: `/v3/users/7eafa4211ada43a0b09036090ff902f7` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:37.166671 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050ad536-3d84-4215-adc2-f1bdcf55c7c2 None admin] RBAC: Authorizing `identity:get_user(user_id=7eafa4211ada43a0b09036090ff902f7)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:37.168326 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050ad536-3d84-4215-adc2-f1bdcf55c7c2 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:37.173341 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 65/131] 10.222.0.89 () {56 vars in 1177 bytes} [Thu Jul 28 17:30:36 2022] GET /identity/v3/users/7eafa4211ada43a0b09036090ff902f7 => generated 275 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:30:37.173341 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050ad536-3d84-4215-adc2-f1bdcf55c7c2 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:37.232817 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917ec394-51b8-49a4-91c4-eabc1fff51f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:37.234114 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917ec394-51b8-49a4-91c4-eabc1fff51f2 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:37.234613 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917ec394-51b8-49a4-91c4-eabc1fff51f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:37.235002 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917ec394-51b8-49a4-91c4-eabc1fff51f2 None admin] PATH_INFO: `/v3/users/7eafa4211ada43a0b09036090ff902f7` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:37.239751 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917ec394-51b8-49a4-91c4-eabc1fff51f2 None admin] RBAC: Authorizing `identity:get_user(user_id=7eafa4211ada43a0b09036090ff902f7)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:37.241446 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917ec394-51b8-49a4-91c4-eabc1fff51f2 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:37.244071 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 67/132] 10.222.0.89 () {56 vars in 1177 bytes} [Thu Jul 28 17:30:36 2022] GET /identity/v3/users/7eafa4211ada43a0b09036090ff902f7 => generated 275 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:30:37.251732 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59c60ea-c370-4e29-8b5a-41bf0cede0bf None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:37.252834 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59c60ea-c370-4e29-8b5a-41bf0cede0bf None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:37.253438 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59c60ea-c370-4e29-8b5a-41bf0cede0bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:37.288645 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-d59c60ea-c370-4e29-8b5a-41bf0cede0bf None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:30:37.291041 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-d59c60ea-c370-4e29-8b5a-41bf0cede0bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:30:37.347680 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050ad536-3d84-4215-adc2-f1bdcf55c7c2 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:37.393486 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193c6cf9-d5df-4142-b764-338816960b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:37.394448 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193c6cf9-d5df-4142-b764-338816960b47 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:37.395000 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193c6cf9-d5df-4142-b764-338816960b47 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:37.395460 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193c6cf9-d5df-4142-b764-338816960b47 None admin] PATH_INFO: `/v3/projects/26339f3896c14d148dd3af450ec9d25f` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:37.399054 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193c6cf9-d5df-4142-b764-338816960b47 None admin] RBAC: Authorizing `identity:get_project(project_id=26339f3896c14d148dd3af450ec9d25f)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:37.400718 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193c6cf9-d5df-4142-b764-338816960b47 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:37.402652 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 66/133] 10.222.0.89 () {56 vars in 1186 bytes} [Thu Jul 28 17:30:37 2022] GET /identity/v3/projects/26339f3896c14d148dd3af450ec9d25f => generated 298 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:30:37.403664 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193c6cf9-d5df-4142-b764-338816960b47 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:37.423710 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:37.445179 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 68/134] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:30:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:30:37.445179 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59c60ea-c370-4e29-8b5a-41bf0cede0bf None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:37.449619 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:37.550025 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193c6cf9-d5df-4142-b764-338816960b47 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:37.589551 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce821b45-e599-4ac4-a310-5d7ea9cc4f62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:37.590767 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce821b45-e599-4ac4-a310-5d7ea9cc4f62 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:37.591375 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce821b45-e599-4ac4-a310-5d7ea9cc4f62 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:37.591859 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce821b45-e599-4ac4-a310-5d7ea9cc4f62 None admin] PATH_INFO: `/v3/projects/26339f3896c14d148dd3af450ec9d25f` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:37.595123 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce821b45-e599-4ac4-a310-5d7ea9cc4f62 None admin] RBAC: Authorizing `identity:get_project(project_id=26339f3896c14d148dd3af450ec9d25f)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:37.596846 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce821b45-e599-4ac4-a310-5d7ea9cc4f62 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:37.599027 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 67/135] 10.222.0.89 () {56 vars in 1186 bytes} [Thu Jul 28 17:30:37 2022] GET /identity/v3/projects/26339f3896c14d148dd3af450ec9d25f => generated 298 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:30:37.601129 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59c60ea-c370-4e29-8b5a-41bf0cede0bf None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:37.603551 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce821b45-e599-4ac4-a310-5d7ea9cc4f62 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:37.660712 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20950bf2-5ce0-46db-9caa-38a616265660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:37.661778 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20950bf2-5ce0-46db-9caa-38a616265660 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:37.662278 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20950bf2-5ce0-46db-9caa-38a616265660 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:37.662765 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20950bf2-5ce0-46db-9caa-38a616265660 None admin] PATH_INFO: `/v3/projects/38d2f8adcb494ba8b566e63d96beceda` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:37.664861 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20950bf2-5ce0-46db-9caa-38a616265660 None admin] RBAC: Authorizing `identity:get_project(project_id=38d2f8adcb494ba8b566e63d96beceda)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:37.666807 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20950bf2-5ce0-46db-9caa-38a616265660 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:37.670433 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 69/136] 10.222.0.89 () {56 vars in 1186 bytes} [Thu Jul 28 17:30:37 2022] GET /identity/v3/projects/38d2f8adcb494ba8b566e63d96beceda => generated 312 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:30:37.672111 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20950bf2-5ce0-46db-9caa-38a616265660 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:37.754279 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:37.771552 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:37.779749 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce821b45-e599-4ac4-a310-5d7ea9cc4f62 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:37.805337 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20950bf2-5ce0-46db-9caa-38a616265660 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:37.831737 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947c8b72-d457-461a-8c0e-8e8caf1a30c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:37.832434 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947c8b72-d457-461a-8c0e-8e8caf1a30c6 None admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:37.832717 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947c8b72-d457-461a-8c0e-8e8caf1a30c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:37.833020 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947c8b72-d457-461a-8c0e-8e8caf1a30c6 None admin] PATH_INFO: `/v3/projects/26339f3896c14d148dd3af450ec9d25f/users/bbc7ab320041413396df1eef4db728ae/roles/73e457231a824ea98c28d22610be640c` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:37.839228 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947c8b72-d457-461a-8c0e-8e8caf1a30c6 None admin] RBAC: Authorizing `identity:create_grant(project_id=26339f3896c14d148dd3af450ec9d25f, user_id=bbc7ab320041413396df1eef4db728ae, role_id=73e457231a824ea98c28d22610be640c)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:37.842188 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947c8b72-d457-461a-8c0e-8e8caf1a30c6 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:37.856632 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2460b0ba-513e-4a9d-afae-17eff8788b14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:37.858154 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2460b0ba-513e-4a9d-afae-17eff8788b14 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:37.858589 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2460b0ba-513e-4a9d-afae-17eff8788b14 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:37.858935 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2460b0ba-513e-4a9d-afae-17eff8788b14 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:37.860639 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2460b0ba-513e-4a9d-afae-17eff8788b14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:37.862290 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2460b0ba-513e-4a9d-afae-17eff8788b14 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:37.863229 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 68/137] 10.222.0.89 () {58 vars in 1439 bytes} [Thu Jul 28 17:30:37 2022] PUT /identity/v3/projects/26339f3896c14d148dd3af450ec9d25f/users/bbc7ab320041413396df1eef4db728ae/roles/73e457231a824ea98c28d22610be640c => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:30:37.864416 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947c8b72-d457-461a-8c0e-8e8caf1a30c6 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:37.878243 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 70/138] 10.222.0.89 () {56 vars in 1108 bytes} [Thu Jul 28 17:30:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:30:37.878243 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2460b0ba-513e-4a9d-afae-17eff8788b14 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:38.023006 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2460b0ba-513e-4a9d-afae-17eff8788b14 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:38.048346 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947c8b72-d457-461a-8c0e-8e8caf1a30c6 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:38.063423 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:38.075428 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270757bd-0010-466d-b5dd-42bf7ea5c901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:38.077392 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270757bd-0010-466d-b5dd-42bf7ea5c901 None admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:38.077896 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270757bd-0010-466d-b5dd-42bf7ea5c901 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:38.078397 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270757bd-0010-466d-b5dd-42bf7ea5c901 None admin] PATH_INFO: `/v3/projects/38d2f8adcb494ba8b566e63d96beceda/users/7eafa4211ada43a0b09036090ff902f7/roles/11f8731756fe4caca13efa61b71b11c5` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:38.082054 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270757bd-0010-466d-b5dd-42bf7ea5c901 None admin] RBAC: Authorizing `identity:create_grant(project_id=38d2f8adcb494ba8b566e63d96beceda, user_id=7eafa4211ada43a0b09036090ff902f7, role_id=11f8731756fe4caca13efa61b71b11c5)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:38.083531 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270757bd-0010-466d-b5dd-42bf7ea5c901 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:38.107783 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 71/139] 10.222.0.89 () {58 vars in 1439 bytes} [Thu Jul 28 17:30:37 2022] PUT /identity/v3/projects/38d2f8adcb494ba8b566e63d96beceda/users/7eafa4211ada43a0b09036090ff902f7/roles/11f8731756fe4caca13efa61b71b11c5 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:30:38.109564 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5838eaab-2250-4e18-85a1-44bb66746a68 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:38.109959 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5838eaab-2250-4e18-85a1-44bb66746a68 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:38.110237 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5838eaab-2250-4e18-85a1-44bb66746a68 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:38.111591 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 72/140] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:30:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:30:38.112568 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5838eaab-2250-4e18-85a1-44bb66746a68 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:38.113447 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcc2b76-20b0-4762-9995-2918ea93ebfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:38.114138 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcc2b76-20b0-4762-9995-2918ea93ebfb None admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:38.115298 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcc2b76-20b0-4762-9995-2918ea93ebfb None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:38.115298 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcc2b76-20b0-4762-9995-2918ea93ebfb None admin] PATH_INFO: `/v3/projects/26339f3896c14d148dd3af450ec9d25f/users/7eafa4211ada43a0b09036090ff902f7/roles/895fd016dffb41e7b0205c9a55a4e0f8` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:38.123860 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcc2b76-20b0-4762-9995-2918ea93ebfb None admin] RBAC: Authorizing `identity:create_grant(project_id=26339f3896c14d148dd3af450ec9d25f, user_id=7eafa4211ada43a0b09036090ff902f7, role_id=895fd016dffb41e7b0205c9a55a4e0f8)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:38.131081 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcc2b76-20b0-4762-9995-2918ea93ebfb None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:38.162618 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 69/141] 10.222.0.89 () {58 vars in 1439 bytes} [Thu Jul 28 17:30:37 2022] PUT /identity/v3/projects/26339f3896c14d148dd3af450ec9d25f/users/7eafa4211ada43a0b09036090ff902f7/roles/895fd016dffb41e7b0205c9a55a4e0f8 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:30:38.167510 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b796d8c0-6b8d-40d0-b83c-8ac70dd7c21f None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:38.167510 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b796d8c0-6b8d-40d0-b83c-8ac70dd7c21f None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:38.167510 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b796d8c0-6b8d-40d0-b83c-8ac70dd7c21f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:38.212613 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-b796d8c0-6b8d-40d0-b83c-8ac70dd7c21f None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:30:38.227075 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-b796d8c0-6b8d-40d0-b83c-8ac70dd7c21f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:30:38.287179 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:38.433347 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5838eaab-2250-4e18-85a1-44bb66746a68 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:38.450224 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 70/142] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:30:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:30:38.486055 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c91e9fa-105c-4a86-9a82-ed335676392b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:38.486887 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c91e9fa-105c-4a86-9a82-ed335676392b None admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:38.487166 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c91e9fa-105c-4a86-9a82-ed335676392b None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:38.487438 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c91e9fa-105c-4a86-9a82-ed335676392b None admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:38.488137 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c91e9fa-105c-4a86-9a82-ed335676392b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:38.492643 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c91e9fa-105c-4a86-9a82-ed335676392b None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:38.492790 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b796d8c0-6b8d-40d0-b83c-8ac70dd7c21f None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:38.549188 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 73/143] 10.222.0.89 () {60 vars in 1132 bytes} [Thu Jul 28 17:30:38 2022] POST /identity/v3/users => generated 283 bytes in 432 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:30:38.667749 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b796d8c0-6b8d-40d0-b83c-8ac70dd7c21f None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:38.755012 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58537e2f-b0ac-4b6a-81ba-bcc1bbc81535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:38.755618 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58537e2f-b0ac-4b6a-81ba-bcc1bbc81535 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:38.755905 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58537e2f-b0ac-4b6a-81ba-bcc1bbc81535 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:38.756183 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58537e2f-b0ac-4b6a-81ba-bcc1bbc81535 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:38.775731 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58537e2f-b0ac-4b6a-81ba-bcc1bbc81535 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:30:38.782061 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-58537e2f-b0ac-4b6a-81ba-bcc1bbc81535 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 17:30:38.782061 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:30:38.782061 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:30:38.782061 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:30:38.782061 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:30:38.782061 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:30:38.782061 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:30:38.782061 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:30:38.782061 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:30:38.782061 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:30:38.782061 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:30:38.782061 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:30:38.782061 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:30:38.782061 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:30:38.782061 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:30:38.782061 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:30:38.783009 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:30:38.783009 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:30:38.783009 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:30:38.783009 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:30:38.783009 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:30:38.783009 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:30:38.783009 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:30:38.783009 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:30:38.783009 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:30:38.783009 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:30:38.783009 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:30:38.783009 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:30:38.783009 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:30:38.783009 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:30:38.783009 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:30:38.783009 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:30:38.783786 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:30:38.783786 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:30:38.783786 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:30:38.783786 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:30:38.783786 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:30:38.783786 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:30:38.783786 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:30:38.783786 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:30:38.783786 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:30:38.783786 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:30:38.783786 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 17:30:38.783786 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:30:38.783786 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 71/144] 10.222.0.89 () {56 vars in 1099 bytes} [Thu Jul 28 17:30:38 2022] GET /identity/v3/roles/reader => generated 84 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:30:38.790465 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c91e9fa-105c-4a86-9a82-ed335676392b None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:38.907860 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:39.015159 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c91e9fa-105c-4a86-9a82-ed335676392b None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:39.053743 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb20cf0-58ef-4397-9822-a2f9984db910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:39.056798 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb20cf0-58ef-4397-9822-a2f9984db910 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:39.057325 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb20cf0-58ef-4397-9822-a2f9984db910 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:39.057816 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb20cf0-58ef-4397-9822-a2f9984db910 None admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:39.058770 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb20cf0-58ef-4397-9822-a2f9984db910 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:30:39.059332 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb20cf0-58ef-4397-9822-a2f9984db910 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:39.060980 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb20cf0-58ef-4397-9822-a2f9984db910 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:39.070719 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 74/145] 10.222.0.89 () {56 vars in 1101 bytes} [Thu Jul 28 17:30:38 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:30:39.080899 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58537e2f-b0ac-4b6a-81ba-bcc1bbc81535 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:39.105709 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:39.259986 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58537e2f-b0ac-4b6a-81ba-bcc1bbc81535 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:39.307421 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a31e5d-b19b-4080-979b-8dbb890cc1f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:39.308467 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a31e5d-b19b-4080-979b-8dbb890cc1f4 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:39.308956 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a31e5d-b19b-4080-979b-8dbb890cc1f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:39.309450 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a31e5d-b19b-4080-979b-8dbb890cc1f4 None admin] PATH_INFO: `/v3/users/5c88f62bf45d45afade134bc5ff1e000` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:39.312798 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a31e5d-b19b-4080-979b-8dbb890cc1f4 None admin] RBAC: Authorizing `identity:get_user(user_id=5c88f62bf45d45afade134bc5ff1e000)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:39.314582 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a31e5d-b19b-4080-979b-8dbb890cc1f4 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:39.316791 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 72/146] 10.222.0.89 () {56 vars in 1177 bytes} [Thu Jul 28 17:30:39 2022] GET /identity/v3/users/5c88f62bf45d45afade134bc5ff1e000 => generated 289 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:30:39.327416 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb20cf0-58ef-4397-9822-a2f9984db910 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:39.434189 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ba7bba-3b51-4727-9e79-35b39429bce3 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:39.436098 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ba7bba-3b51-4727-9e79-35b39429bce3 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:39.437348 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ba7bba-3b51-4727-9e79-35b39429bce3 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:39.439553 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 73/147] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:30:39 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:30:39.451761 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf599ef0-e9bb-4ccd-9fbb-dac971876278 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:39.452945 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf599ef0-e9bb-4ccd-9fbb-dac971876278 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:39.453556 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf599ef0-e9bb-4ccd-9fbb-dac971876278 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:39.486379 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-cf599ef0-e9bb-4ccd-9fbb-dac971876278 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:30:39.487542 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-cf599ef0-e9bb-4ccd-9fbb-dac971876278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:30:39.539521 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb20cf0-58ef-4397-9822-a2f9984db910 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:39.556471 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:39.613064 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6a5a2c-15ac-4cd5-bae7-b532f5ebe20c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:39.613476 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6a5a2c-15ac-4cd5-bae7-b532f5ebe20c None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:39.613756 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6a5a2c-15ac-4cd5-bae7-b532f5ebe20c None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:39.614011 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6a5a2c-15ac-4cd5-bae7-b532f5ebe20c None admin] PATH_INFO: `/v3/projects/26339f3896c14d148dd3af450ec9d25f` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:39.617913 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6a5a2c-15ac-4cd5-bae7-b532f5ebe20c None admin] RBAC: Authorizing `identity:get_project(project_id=26339f3896c14d148dd3af450ec9d25f)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:39.619759 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6a5a2c-15ac-4cd5-bae7-b532f5ebe20c None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:39.621550 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 75/148] 10.222.0.89 () {56 vars in 1186 bytes} [Thu Jul 28 17:30:39 2022] GET /identity/v3/projects/26339f3896c14d148dd3af450ec9d25f => generated 298 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:30:39.629495 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6a5a2c-15ac-4cd5-bae7-b532f5ebe20c None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:39.746057 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 74/149] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:30:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:30:39.787583 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf599ef0-e9bb-4ccd-9fbb-dac971876278 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:39.821922 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6a5a2c-15ac-4cd5-bae7-b532f5ebe20c None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:39.829486 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:39.908151 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3484d5a-4edb-4df3-bbca-dc3a143c3be3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:39.909115 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3484d5a-4edb-4df3-bbca-dc3a143c3be3 None admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:39.909590 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3484d5a-4edb-4df3-bbca-dc3a143c3be3 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:39.910023 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3484d5a-4edb-4df3-bbca-dc3a143c3be3 None admin] PATH_INFO: `/v3/projects/26339f3896c14d148dd3af450ec9d25f/users/5c88f62bf45d45afade134bc5ff1e000/roles/466df921c2db4456b37637c23382d92a` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:39.918068 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3484d5a-4edb-4df3-bbca-dc3a143c3be3 None admin] RBAC: Authorizing `identity:create_grant(project_id=26339f3896c14d148dd3af450ec9d25f, user_id=5c88f62bf45d45afade134bc5ff1e000, role_id=466df921c2db4456b37637c23382d92a)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:39.919886 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3484d5a-4edb-4df3-bbca-dc3a143c3be3 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:39.954507 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 76/150] 10.222.0.89 () {58 vars in 1439 bytes} [Thu Jul 28 17:30:39 2022] PUT /identity/v3/projects/26339f3896c14d148dd3af450ec9d25f/users/5c88f62bf45d45afade134bc5ff1e000/roles/466df921c2db4456b37637c23382d92a => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:30:40.147593 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf599ef0-e9bb-4ccd-9fbb-dac971876278 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:40.215451 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f1e8ee-e7e7-4880-bed4-7acaf800c9f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:40.216141 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f1e8ee-e7e7-4880-bed4-7acaf800c9f0 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:40.216433 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f1e8ee-e7e7-4880-bed4-7acaf800c9f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:40.216714 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f1e8ee-e7e7-4880-bed4-7acaf800c9f0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:40.241909 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f1e8ee-e7e7-4880-bed4-7acaf800c9f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:30:40.247267 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-83f1e8ee-e7e7-4880-bed4-7acaf800c9f0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 17:30:40.247267 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:30:40.247267 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:30:40.247267 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:30:40.247267 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:30:40.247267 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:30:40.247267 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:30:40.247267 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:30:40.247267 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:30:40.247267 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:30:40.247267 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:30:40.247267 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:30:40.247267 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:30:40.247267 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:30:40.247267 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:30:40.247267 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:30:40.248167 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:30:40.248167 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:30:40.248167 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:30:40.248167 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:30:40.248167 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:30:40.248167 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:30:40.248167 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:30:40.248167 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:30:40.248167 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:30:40.248167 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:30:40.248167 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:30:40.248167 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:30:40.248167 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:30:40.248167 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:30:40.248167 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:30:40.248167 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:30:40.249509 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:30:40.249509 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:30:40.249509 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:30:40.249509 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:30:40.249509 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:30:40.249509 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:30:40.249509 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:30:40.249509 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:30:40.249509 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:30:40.249509 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:30:40.249509 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 17:30:40.249509 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:30:40.249509 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 75/151] 10.222.0.89 () {56 vars in 1099 bytes} [Thu Jul 28 17:30:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 463 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:30:40.258373 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3484d5a-4edb-4df3-bbca-dc3a143c3be3 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:40.342434 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:40.493651 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3484d5a-4edb-4df3-bbca-dc3a143c3be3 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:40.563059 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd3cf0b-35d8-44b3-8d6a-addf9cc865c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:40.563694 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd3cf0b-35d8-44b3-8d6a-addf9cc865c4 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:40.563979 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd3cf0b-35d8-44b3-8d6a-addf9cc865c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:40.564255 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd3cf0b-35d8-44b3-8d6a-addf9cc865c4 None admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:40.564985 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd3cf0b-35d8-44b3-8d6a-addf9cc865c4 None admin] RBAC: Adding query filter params (name=member) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:30:40.565339 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd3cf0b-35d8-44b3-8d6a-addf9cc865c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:40.566740 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd3cf0b-35d8-44b3-8d6a-addf9cc865c4 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:40.586989 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 77/152] 10.222.0.89 () {56 vars in 1101 bytes} [Thu Jul 28 17:30:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:30:40.593706 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f1e8ee-e7e7-4880-bed4-7acaf800c9f0 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:40.749577 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:40.805065 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f1e8ee-e7e7-4880-bed4-7acaf800c9f0 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:40.871709 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616454ac-4307-40cf-86ff-553210f746e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:40.872379 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616454ac-4307-40cf-86ff-553210f746e2 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:40.872663 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616454ac-4307-40cf-86ff-553210f746e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:40.872936 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616454ac-4307-40cf-86ff-553210f746e2 None admin] PATH_INFO: `/v3/users/7eafa4211ada43a0b09036090ff902f7` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:40.874745 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616454ac-4307-40cf-86ff-553210f746e2 None admin] RBAC: Authorizing `identity:get_user(user_id=7eafa4211ada43a0b09036090ff902f7)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:40.876309 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616454ac-4307-40cf-86ff-553210f746e2 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:40.881179 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 76/153] 10.222.0.89 () {56 vars in 1177 bytes} [Thu Jul 28 17:30:40 2022] GET /identity/v3/users/7eafa4211ada43a0b09036090ff902f7 => generated 275 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:30:40.885180 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd3cf0b-35d8-44b3-8d6a-addf9cc865c4 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:41.110688 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:41.116285 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd3cf0b-35d8-44b3-8d6a-addf9cc865c4 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:41.194732 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1af08ef-5618-4f49-b7d8-09b0ced64905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:41.195372 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1af08ef-5618-4f49-b7d8-09b0ced64905 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:41.195654 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1af08ef-5618-4f49-b7d8-09b0ced64905 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:41.195933 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1af08ef-5618-4f49-b7d8-09b0ced64905 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:41.378416 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1af08ef-5618-4f49-b7d8-09b0ced64905 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:41.381237 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1af08ef-5618-4f49-b7d8-09b0ced64905 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:41.547843 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 78/154] 10.222.0.89 () {58 vars in 1282 bytes} [Thu Jul 28 17:30:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 665 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 17:30:41.550004 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:41.554512 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616454ac-4307-40cf-86ff-553210f746e2 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:41.722226 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616454ac-4307-40cf-86ff-553210f746e2 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:41.808751 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff7c894-4002-4145-90ce-55c62491e7af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:41.809446 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff7c894-4002-4145-90ce-55c62491e7af None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:41.810144 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff7c894-4002-4145-90ce-55c62491e7af None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:41.810144 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff7c894-4002-4145-90ce-55c62491e7af None admin] PATH_INFO: `/v3/projects/26339f3896c14d148dd3af450ec9d25f` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:41.811335 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff7c894-4002-4145-90ce-55c62491e7af None admin] RBAC: Authorizing `identity:get_project(project_id=26339f3896c14d148dd3af450ec9d25f)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:41.814807 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff7c894-4002-4145-90ce-55c62491e7af None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:41.823110 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 77/155] 10.222.0.89 () {56 vars in 1186 bytes} [Thu Jul 28 17:30:41 2022] GET /identity/v3/projects/26339f3896c14d148dd3af450ec9d25f => generated 298 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:30:41.832236 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1af08ef-5618-4f49-b7d8-09b0ced64905 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:41.873613 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfbea8e-c505-440b-83f4-35035dc08edf None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:41.873955 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfbea8e-c505-440b-83f4-35035dc08edf None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:41.874246 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfbea8e-c505-440b-83f4-35035dc08edf None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:41.875859 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 78/156] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:30:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 17:30:41.886506 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bbe7cf-e1a8-47f7-a73c-70972e858a9f None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:41.886790 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bbe7cf-e1a8-47f7-a73c-70972e858a9f None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:41.887071 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bbe7cf-e1a8-47f7-a73c-70972e858a9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:41.897894 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:41.930755 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-47bbe7cf-e1a8-47f7-a73c-70972e858a9f None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:30:41.932002 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-47bbe7cf-e1a8-47f7-a73c-70972e858a9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:30:42.110839 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1af08ef-5618-4f49-b7d8-09b0ced64905 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:42.111143 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 79/157] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:30:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 17:30:42.198220 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bbe7cf-e1a8-47f7-a73c-70972e858a9f None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:42.218516 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a313f0c8-14ea-41c1-bc70-4310e3e5f6f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:42.219348 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a313f0c8-14ea-41c1-bc70-4310e3e5f6f5 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:42.219634 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a313f0c8-14ea-41c1-bc70-4310e3e5f6f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:42.219912 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a313f0c8-14ea-41c1-bc70-4310e3e5f6f5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:42.220726 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a313f0c8-14ea-41c1-bc70-4310e3e5f6f5 None admin] RBAC: Adding query filter params (role.id=11f8731756fe4caca13efa61b71b11c5, scope.project.id=26339f3896c14d148dd3af450ec9d25f, user.id=7eafa4211ada43a0b09036090ff902f7) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:30:42.221042 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a313f0c8-14ea-41c1-bc70-4310e3e5f6f5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:42.223135 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a313f0c8-14ea-41c1-bc70-4310e3e5f6f5 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:42.251615 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 79/158] 10.222.0.89 () {56 vars in 1374 bytes} [Thu Jul 28 17:30:41 2022] GET /identity/v3/role_assignments?user.id=7eafa4211ada43a0b09036090ff902f7&scope.project.id=26339f3896c14d148dd3af450ec9d25f&role.id=11f8731756fe4caca13efa61b71b11c5 => generated 608 bytes in 420 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:30:42.379415 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b34bde-1c9c-4b5c-aa30-4539ebdae8f9 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:42.379989 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b34bde-1c9c-4b5c-aa30-4539ebdae8f9 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:42.379989 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b34bde-1c9c-4b5c-aa30-4539ebdae8f9 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:42.381540 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 80/159] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:30:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:30:42.386552 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7d1efc-dda5-491f-a0a2-e8dcc3969998 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:42.386840 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7d1efc-dda5-491f-a0a2-e8dcc3969998 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:42.387119 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7d1efc-dda5-491f-a0a2-e8dcc3969998 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:42.419303 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-6e7d1efc-dda5-491f-a0a2-e8dcc3969998 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:30:42.423573 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-6e7d1efc-dda5-491f-a0a2-e8dcc3969998 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:30:42.507462 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bbe7cf-e1a8-47f7-a73c-70972e858a9f None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:42.592679 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:42.598150 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3cb78c-5a9e-4d63-a54f-b0d0f974b13a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:42.598837 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3cb78c-5a9e-4d63-a54f-b0d0f974b13a None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:42.599125 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3cb78c-5a9e-4d63-a54f-b0d0f974b13a None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:42.599431 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3cb78c-5a9e-4d63-a54f-b0d0f974b13a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:42.610928 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:42.620291 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 81/160] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:30:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:30:42.621992 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a6513d-4f59-4c2b-9a46-b12e801448b8 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:42.622488 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a6513d-4f59-4c2b-9a46-b12e801448b8 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:42.622982 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a6513d-4f59-4c2b-9a46-b12e801448b8 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:42.629183 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 82/161] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:30:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:30:42.633242 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3cb78c-5a9e-4d63-a54f-b0d0f974b13a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:30:42.634689 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa55cfa2-ca7e-493d-a9ee-17bf6a54e425 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:42.635182 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa55cfa2-ca7e-493d-a9ee-17bf6a54e425 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:42.635656 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa55cfa2-ca7e-493d-a9ee-17bf6a54e425 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:42.649543 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-1d3cb78c-5a9e-4d63-a54f-b0d0f974b13a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:30:42.649543 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:30:42.649543 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:30:42.649543 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:30:42.649543 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:30:42.649543 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:30:42.649543 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:30:42.649543 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:30:42.649543 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:30:42.649543 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:30:42.649543 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:30:42.649543 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:30:42.649543 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:30:42.649543 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:30:42.649543 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:30:42.649543 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:30:42.650438 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:30:42.650438 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:30:42.650438 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:30:42.650438 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:30:42.650438 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:30:42.650438 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:30:42.650438 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:30:42.650438 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:30:42.650438 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:30:42.650438 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:30:42.650438 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:30:42.650438 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:30:42.650438 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:30:42.650438 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:30:42.650438 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:30:42.650438 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:30:42.651195 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:30:42.651195 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:30:42.651195 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:30:42.651195 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:30:42.651195 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:30:42.651195 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:30:42.651195 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:30:42.651195 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:30:42.651195 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:30:42.651195 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:30:42.651195 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:30:42.651195 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:30:42.661191 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 80/162] 10.222.0.89 () {56 vars in 1096 bytes} [Thu Jul 28 17:30:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 456 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:30:42.672914 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3cb78c-5a9e-4d63-a54f-b0d0f974b13a None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:42.679632 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-fa55cfa2-ca7e-493d-a9ee-17bf6a54e425 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:30:42.680885 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-fa55cfa2-ca7e-493d-a9ee-17bf6a54e425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:30:42.829719 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 83/163] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:30:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:30:42.830658 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa55cfa2-ca7e-493d-a9ee-17bf6a54e425 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:42.880544 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3cb78c-5a9e-4d63-a54f-b0d0f974b13a None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:42.970573 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5918ce1-8372-4402-bef0-a7db95c0b17d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:42.971246 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5918ce1-8372-4402-bef0-a7db95c0b17d None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:42.971526 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5918ce1-8372-4402-bef0-a7db95c0b17d None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:42.971799 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5918ce1-8372-4402-bef0-a7db95c0b17d None admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:42.972523 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5918ce1-8372-4402-bef0-a7db95c0b17d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:30:42.972827 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5918ce1-8372-4402-bef0-a7db95c0b17d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:42.974413 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5918ce1-8372-4402-bef0-a7db95c0b17d None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:42.989080 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 81/164] 10.222.0.89 () {56 vars in 1099 bytes} [Thu Jul 28 17:30:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:30:42.990308 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5918ce1-8372-4402-bef0-a7db95c0b17d None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:43.010667 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa55cfa2-ca7e-493d-a9ee-17bf6a54e425 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:43.054873 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:43.084253 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b426c05-5fbb-44cc-a22d-eab6b9695844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:43.085288 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b426c05-5fbb-44cc-a22d-eab6b9695844 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:43.085804 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b426c05-5fbb-44cc-a22d-eab6b9695844 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:43.086301 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b426c05-5fbb-44cc-a22d-eab6b9695844 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:43.098220 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:43.107878 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b426c05-5fbb-44cc-a22d-eab6b9695844 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:30:43.121923 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-4b426c05-5fbb-44cc-a22d-eab6b9695844 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 17:30:43.121923 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:30:43.121923 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:30:43.121923 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:30:43.121923 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:30:43.121923 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:30:43.121923 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:30:43.121923 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:30:43.121923 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:30:43.121923 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:30:43.121923 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:30:43.121923 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:30:43.121923 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:30:43.121923 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:30:43.121923 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:30:43.121923 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:30:43.122908 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:30:43.122908 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:30:43.122908 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:30:43.122908 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:30:43.122908 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:30:43.122908 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:30:43.122908 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:30:43.122908 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:30:43.122908 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:30:43.122908 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:30:43.122908 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:30:43.122908 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( Jul 28 17:30:43.122908 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:30:43.122908 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:30:43.122908 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:30:43.122908 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:30:43.123684 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:30:43.123684 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:30:43.123684 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:30:43.123684 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:30:43.123684 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:30:43.123684 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:30:43.123684 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:30:43.123684 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:30:43.123684 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:30:43.123684 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:30:43.123684 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 17:30:43.123684 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:30:43.126255 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 84/165] 10.222.0.89 () {56 vars in 1099 bytes} [Thu Jul 28 17:30:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:30:43.127423 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b426c05-5fbb-44cc-a22d-eab6b9695844 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:43.168575 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5918ce1-8372-4402-bef0-a7db95c0b17d None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:43.242640 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a76b42-685b-43bf-a21f-99a3d7c82efb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:43.243525 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a76b42-685b-43bf-a21f-99a3d7c82efb None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:43.244215 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a76b42-685b-43bf-a21f-99a3d7c82efb None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:43.244627 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a76b42-685b-43bf-a21f-99a3d7c82efb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:43.270137 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a76b42-685b-43bf-a21f-99a3d7c82efb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:30:43.276018 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-d4a76b42-685b-43bf-a21f-99a3d7c82efb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 17:30:43.276018 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:30:43.276018 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:30:43.276018 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:30:43.276018 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:30:43.276018 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:30:43.276018 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:30:43.276018 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:30:43.276018 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:30:43.276018 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:30:43.276018 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:30:43.276018 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:30:43.276018 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:30:43.276018 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:30:43.276018 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:30:43.276018 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:30:43.276870 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:30:43.276870 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:30:43.276870 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:30:43.276870 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:30:43.276870 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:30:43.276870 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:30:43.276870 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:30:43.276870 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:30:43.276870 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:30:43.276870 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:30:43.276870 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:30:43.276870 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:30:43.276870 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:30:43.276870 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:30:43.276870 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:30:43.276870 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:30:43.277679 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:30:43.277679 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:30:43.277679 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:30:43.277679 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:30:43.277679 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:30:43.277679 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:30:43.277679 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:30:43.277679 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:30:43.277679 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:30:43.277679 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:30:43.277679 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 17:30:43.277679 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:30:43.279522 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 82/166] 10.222.0.89 () {56 vars in 1114 bytes} [Thu Jul 28 17:30:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:30:43.289235 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a76b42-685b-43bf-a21f-99a3d7c82efb None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:43.337150 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b426c05-5fbb-44cc-a22d-eab6b9695844 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:43.409079 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7c238e-77ab-48af-bef2-dfe93c46d47a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:43.410526 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7c238e-77ab-48af-bef2-dfe93c46d47a None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:43.411011 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7c238e-77ab-48af-bef2-dfe93c46d47a None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:43.411498 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7c238e-77ab-48af-bef2-dfe93c46d47a None admin] PATH_INFO: `/v3/users/bbc7ab320041413396df1eef4db728ae` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:43.413001 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7c238e-77ab-48af-bef2-dfe93c46d47a None admin] RBAC: Authorizing `identity:get_user(user_id=bbc7ab320041413396df1eef4db728ae)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:43.414806 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7c238e-77ab-48af-bef2-dfe93c46d47a None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:43.417018 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 85/167] 10.222.0.89 () {56 vars in 1177 bytes} [Thu Jul 28 17:30:43 2022] GET /identity/v3/users/bbc7ab320041413396df1eef4db728ae => generated 247 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:30:43.418010 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7c238e-77ab-48af-bef2-dfe93c46d47a None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:43.478631 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:43.540741 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7c238e-77ab-48af-bef2-dfe93c46d47a None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:43.560173 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a76b42-685b-43bf-a21f-99a3d7c82efb None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:43.581541 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f7b089-0f65-4a37-b9d1-c0c4281eda45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:43.582535 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f7b089-0f65-4a37-b9d1-c0c4281eda45 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:43.583019 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f7b089-0f65-4a37-b9d1-c0c4281eda45 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:43.583507 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f7b089-0f65-4a37-b9d1-c0c4281eda45 None admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:43.584424 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f7b089-0f65-4a37-b9d1-c0c4281eda45 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:30:43.584929 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f7b089-0f65-4a37-b9d1-c0c4281eda45 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:43.586543 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f7b089-0f65-4a37-b9d1-c0c4281eda45 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:43.595646 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 86/168] 10.222.0.89 () {56 vars in 1111 bytes} [Thu Jul 28 17:30:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:30:43.597213 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d98d96-4fb7-49b3-9213-c42493c4eba4 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:43.597731 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d98d96-4fb7-49b3-9213-c42493c4eba4 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:43.598243 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d98d96-4fb7-49b3-9213-c42493c4eba4 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:43.599667 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 87/169] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:30:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:30:43.600697 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d98d96-4fb7-49b3-9213-c42493c4eba4 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:43.607341 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:43.635320 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30912466-950d-4300-8d04-d02e31599ff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:43.636186 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30912466-950d-4300-8d04-d02e31599ff3 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:43.636611 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30912466-950d-4300-8d04-d02e31599ff3 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:43.637028 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30912466-950d-4300-8d04-d02e31599ff3 None admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:43.637956 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30912466-950d-4300-8d04-d02e31599ff3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:30:43.638441 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30912466-950d-4300-8d04-d02e31599ff3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:43.640612 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30912466-950d-4300-8d04-d02e31599ff3 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:43.653658 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 83/170] 10.222.0.89 () {56 vars in 1101 bytes} [Thu Jul 28 17:30:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 373 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:30:43.655981 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c91cf8-501a-4b80-b48b-5dd344709176 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:43.656454 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c91cf8-501a-4b80-b48b-5dd344709176 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:43.656930 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c91cf8-501a-4b80-b48b-5dd344709176 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:43.676999 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-55c91cf8-501a-4b80-b48b-5dd344709176 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:30:43.678540 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-55c91cf8-501a-4b80-b48b-5dd344709176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:30:43.725064 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d98d96-4fb7-49b3-9213-c42493c4eba4 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:43.740308 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:43.758383 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 84/171] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:30:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:30:43.761058 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c91cf8-501a-4b80-b48b-5dd344709176 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:43.805959 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fb495c-e35c-4219-b4ec-12b1374df9ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:43.806805 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fb495c-e35c-4219-b4ec-12b1374df9ec None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:43.807231 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fb495c-e35c-4219-b4ec-12b1374df9ec None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:43.807635 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fb495c-e35c-4219-b4ec-12b1374df9ec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:43.956655 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c91cf8-501a-4b80-b48b-5dd344709176 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:43.966091 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fb495c-e35c-4219-b4ec-12b1374df9ec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:43.968873 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fb495c-e35c-4219-b4ec-12b1374df9ec None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:43.969695 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:44.003599 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bc7dbd-db48-4211-a56b-5c8495ea7f4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:44.004910 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bc7dbd-db48-4211-a56b-5c8495ea7f4a None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:44.005641 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bc7dbd-db48-4211-a56b-5c8495ea7f4a None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:44.006089 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bc7dbd-db48-4211-a56b-5c8495ea7f4a None admin] PATH_INFO: `/v3/users/7eafa4211ada43a0b09036090ff902f7` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:44.007990 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bc7dbd-db48-4211-a56b-5c8495ea7f4a None admin] RBAC: Authorizing `identity:get_user(user_id=7eafa4211ada43a0b09036090ff902f7)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:44.010099 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bc7dbd-db48-4211-a56b-5c8495ea7f4a None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:44.017484 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 85/172] 10.222.0.89 () {56 vars in 1177 bytes} [Thu Jul 28 17:30:43 2022] GET /identity/v3/users/7eafa4211ada43a0b09036090ff902f7 => generated 275 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:30:44.019643 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bc7dbd-db48-4211-a56b-5c8495ea7f4a None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:44.109447 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 88/173] 10.222.0.89 () {58 vars in 1282 bytes} [Thu Jul 28 17:30:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 509 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 17:30:44.115310 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fb495c-e35c-4219-b4ec-12b1374df9ec None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:44.134774 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:44.160885 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bc7dbd-db48-4211-a56b-5c8495ea7f4a None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:44.200889 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287b4370-87ed-48b0-95f3-84677d9c0bac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:44.201960 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287b4370-87ed-48b0-95f3-84677d9c0bac None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:44.202447 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287b4370-87ed-48b0-95f3-84677d9c0bac None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:44.202919 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287b4370-87ed-48b0-95f3-84677d9c0bac None admin] PATH_INFO: `/v3/users/7eafa4211ada43a0b09036090ff902f7` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:44.204364 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287b4370-87ed-48b0-95f3-84677d9c0bac None admin] RBAC: Authorizing `identity:get_user(user_id=7eafa4211ada43a0b09036090ff902f7)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:44.206060 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287b4370-87ed-48b0-95f3-84677d9c0bac None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:44.208136 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 86/174] 10.222.0.89 () {56 vars in 1177 bytes} [Thu Jul 28 17:30:44 2022] GET /identity/v3/users/7eafa4211ada43a0b09036090ff902f7 => generated 275 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:30:44.209759 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5a5e3e-de87-438f-a8c0-b25874a9f9ae None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:44.210318 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5a5e3e-de87-438f-a8c0-b25874a9f9ae None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:44.210802 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5a5e3e-de87-438f-a8c0-b25874a9f9ae None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:44.212271 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 87/175] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:30:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:30:44.218253 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5a5e3e-de87-438f-a8c0-b25874a9f9ae None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:44.258052 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fb495c-e35c-4219-b4ec-12b1374df9ec None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:44.272463 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:44.310644 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6cdd94-d8c7-43db-a65f-9f52f5e1d142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:44.311680 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6cdd94-d8c7-43db-a65f-9f52f5e1d142 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:44.313021 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6cdd94-d8c7-43db-a65f-9f52f5e1d142 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:44.313551 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6cdd94-d8c7-43db-a65f-9f52f5e1d142 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:44.326865 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6cdd94-d8c7-43db-a65f-9f52f5e1d142 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:30:44.332571 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-ec6cdd94-d8c7-43db-a65f-9f52f5e1d142 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:30:44.332571 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:30:44.332571 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:30:44.332571 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:30:44.332571 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:30:44.332571 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:30:44.332571 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:30:44.332571 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:30:44.332571 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:30:44.332571 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:30:44.332571 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:30:44.332571 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:30:44.332571 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:30:44.332571 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:30:44.332571 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:30:44.332571 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:30:44.333489 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:30:44.333489 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:30:44.333489 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:30:44.333489 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:30:44.333489 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:30:44.333489 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:30:44.333489 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:30:44.333489 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:30:44.333489 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:30:44.333489 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:30:44.333489 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:30:44.333489 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( Jul 28 17:30:44.333489 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:30:44.333489 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:30:44.333489 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:30:44.333489 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:30:44.334285 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:30:44.334285 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:30:44.334285 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:30:44.334285 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:30:44.334285 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:30:44.334285 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:30:44.334285 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:30:44.334285 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:30:44.334285 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:30:44.334285 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:30:44.334285 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:30:44.334285 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:30:44.334285 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 89/176] 10.222.0.89 () {56 vars in 1096 bytes} [Thu Jul 28 17:30:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:30:44.335617 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6cdd94-d8c7-43db-a65f-9f52f5e1d142 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:44.349906 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5a5e3e-de87-438f-a8c0-b25874a9f9ae None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:44.390004 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae487a94-4476-483d-9a16-11f86454feca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:44.390589 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae487a94-4476-483d-9a16-11f86454feca None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:44.390865 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae487a94-4476-483d-9a16-11f86454feca None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:44.391133 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae487a94-4476-483d-9a16-11f86454feca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:44.421745 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:44.448415 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6cdd94-d8c7-43db-a65f-9f52f5e1d142 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:44.460679 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae487a94-4476-483d-9a16-11f86454feca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:44.462192 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae487a94-4476-483d-9a16-11f86454feca None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:44.484583 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299bcee2-dd46-49a2-aa83-be4fe33c36e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:44.485246 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299bcee2-dd46-49a2-aa83-be4fe33c36e5 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:44.485529 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299bcee2-dd46-49a2-aa83-be4fe33c36e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:44.485803 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299bcee2-dd46-49a2-aa83-be4fe33c36e5 None admin] PATH_INFO: `/v3/projects/26339f3896c14d148dd3af450ec9d25f` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:44.487091 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299bcee2-dd46-49a2-aa83-be4fe33c36e5 None admin] RBAC: Authorizing `identity:get_project(project_id=26339f3896c14d148dd3af450ec9d25f)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:44.488558 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299bcee2-dd46-49a2-aa83-be4fe33c36e5 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:44.490387 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 90/177] 10.222.0.89 () {56 vars in 1186 bytes} [Thu Jul 28 17:30:44 2022] GET /identity/v3/projects/26339f3896c14d148dd3af450ec9d25f => generated 298 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:30:44.491738 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5ce989-2a1b-4afd-b1c8-2b138a9b708c None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:44.491862 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5ce989-2a1b-4afd-b1c8-2b138a9b708c None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:44.492066 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5ce989-2a1b-4afd-b1c8-2b138a9b708c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:44.510208 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-9f5ce989-2a1b-4afd-b1c8-2b138a9b708c None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:30:44.511427 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-9f5ce989-2a1b-4afd-b1c8-2b138a9b708c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:30:44.511814 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:44.569098 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 88/178] 10.222.0.89 () {58 vars in 1282 bytes} [Thu Jul 28 17:30:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 17:30:44.569979 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae487a94-4476-483d-9a16-11f86454feca None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:44.579463 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 91/179] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:30:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:30:44.580697 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1165c9-d37f-4420-b0a0-5d672e8a7c71 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:44.580990 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1165c9-d37f-4420-b0a0-5d672e8a7c71 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:44.581309 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1165c9-d37f-4420-b0a0-5d672e8a7c71 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:44.582652 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 92/180] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:30:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:30:44.583284 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1165c9-d37f-4420-b0a0-5d672e8a7c71 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:44.654888 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:44.685266 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae487a94-4476-483d-9a16-11f86454feca None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:44.686876 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1165c9-d37f-4420-b0a0-5d672e8a7c71 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:44.720514 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badcfd48-0a3e-4b6b-9579-d6abaeae968c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:44.721144 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badcfd48-0a3e-4b6b-9579-d6abaeae968c None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:44.721470 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badcfd48-0a3e-4b6b-9579-d6abaeae968c None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:44.721627 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583e7a15-ec7b-40cb-9742-162feb262d40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:44.721733 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badcfd48-0a3e-4b6b-9579-d6abaeae968c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:44.722184 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583e7a15-ec7b-40cb-9742-162feb262d40 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:44.722463 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583e7a15-ec7b-40cb-9742-162feb262d40 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:44.722732 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583e7a15-ec7b-40cb-9742-162feb262d40 None admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:44.723455 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583e7a15-ec7b-40cb-9742-162feb262d40 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:30:44.723755 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583e7a15-ec7b-40cb-9742-162feb262d40 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:44.725135 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583e7a15-ec7b-40cb-9742-162feb262d40 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:44.734281 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 93/181] 10.222.0.89 () {56 vars in 1099 bytes} [Thu Jul 28 17:30:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:30:44.735655 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aedb0f-18a8-4362-9d14-806e80995559 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:44.735969 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aedb0f-18a8-4362-9d14-806e80995559 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:44.736242 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aedb0f-18a8-4362-9d14-806e80995559 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:44.737628 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 94/182] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:30:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:30:44.738236 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84aedb0f-18a8-4362-9d14-806e80995559 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:44.744238 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:44.799767 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badcfd48-0a3e-4b6b-9579-d6abaeae968c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:44.801282 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badcfd48-0a3e-4b6b-9579-d6abaeae968c None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:44.843471 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84aedb0f-18a8-4362-9d14-806e80995559 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:44.876942 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd074b16-0cb1-45b1-89b7-99e47913a504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:44.877581 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd074b16-0cb1-45b1-89b7-99e47913a504 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:44.877865 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd074b16-0cb1-45b1-89b7-99e47913a504 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:44.878139 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd074b16-0cb1-45b1-89b7-99e47913a504 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:44.878926 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd074b16-0cb1-45b1-89b7-99e47913a504 None admin] RBAC: Adding query filter params (role.id=73e457231a824ea98c28d22610be640c, scope.project.id=26339f3896c14d148dd3af450ec9d25f, user.id=bbc7ab320041413396df1eef4db728ae) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:30:44.879234 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd074b16-0cb1-45b1-89b7-99e47913a504 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:44.880670 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd074b16-0cb1-45b1-89b7-99e47913a504 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:44.882476 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:44.887346 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 95/183] 10.222.0.89 () {56 vars in 1374 bytes} [Thu Jul 28 17:30:44 2022] GET /identity/v3/role_assignments?user.id=bbc7ab320041413396df1eef4db728ae&scope.project.id=26339f3896c14d148dd3af450ec9d25f&role.id=73e457231a824ea98c28d22610be640c => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:30:44.898546 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd074b16-0cb1-45b1-89b7-99e47913a504 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:44.909923 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 89/184] 10.222.0.89 () {58 vars in 1282 bytes} [Thu Jul 28 17:30:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 17:30:44.911886 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4361f57d-46ec-4ac6-9ef1-c357e00e81d6 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:44.912176 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4361f57d-46ec-4ac6-9ef1-c357e00e81d6 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:44.912820 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4361f57d-46ec-4ac6-9ef1-c357e00e81d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:44.933117 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-4361f57d-46ec-4ac6-9ef1-c357e00e81d6 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:30:44.934577 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-4361f57d-46ec-4ac6-9ef1-c357e00e81d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:30:44.995720 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:45.046724 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 90/185] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:30:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:30:45.049278 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4361f57d-46ec-4ac6-9ef1-c357e00e81d6 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:45.060324 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd074b16-0cb1-45b1-89b7-99e47913a504 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:45.118682 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59e1b36-247a-42ea-a504-3d9e8f816e32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:45.120696 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59e1b36-247a-42ea-a504-3d9e8f816e32 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:45.121195 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59e1b36-247a-42ea-a504-3d9e8f816e32 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:45.122140 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59e1b36-247a-42ea-a504-3d9e8f816e32 None admin] PATH_INFO: `/v3/projects/26339f3896c14d148dd3af450ec9d25f` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:45.125349 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59e1b36-247a-42ea-a504-3d9e8f816e32 None admin] RBAC: Authorizing `identity:get_project(project_id=26339f3896c14d148dd3af450ec9d25f)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:45.126080 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59e1b36-247a-42ea-a504-3d9e8f816e32 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:45.128202 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 96/186] 10.222.0.89 () {56 vars in 1186 bytes} [Thu Jul 28 17:30:44 2022] GET /identity/v3/projects/26339f3896c14d148dd3af450ec9d25f => generated 298 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:30:45.128980 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59e1b36-247a-42ea-a504-3d9e8f816e32 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:45.193333 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4361f57d-46ec-4ac6-9ef1-c357e00e81d6 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:45.208781 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:45.230687 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e231d13-e939-4258-80aa-ead4bf136e58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:45.231340 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e231d13-e939-4258-80aa-ead4bf136e58 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:45.231624 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e231d13-e939-4258-80aa-ead4bf136e58 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:45.231899 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e231d13-e939-4258-80aa-ead4bf136e58 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:45.233070 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e231d13-e939-4258-80aa-ead4bf136e58 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:45.234631 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e231d13-e939-4258-80aa-ead4bf136e58 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:45.236630 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 91/187] 10.222.0.89 () {56 vars in 1108 bytes} [Thu Jul 28 17:30:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:30:45.237657 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6df198-3247-4545-beb8-68cc6e95474e None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:45.237938 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6df198-3247-4545-beb8-68cc6e95474e None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:45.238211 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6df198-3247-4545-beb8-68cc6e95474e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:45.248738 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59e1b36-247a-42ea-a504-3d9e8f816e32 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:45.259480 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-bc6df198-3247-4545-beb8-68cc6e95474e None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:30:45.260703 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-bc6df198-3247-4545-beb8-68cc6e95474e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:30:45.288635 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4e2777-9283-407d-a3d7-e9dca87f8c41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:45.289336 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4e2777-9283-407d-a3d7-e9dca87f8c41 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:45.289614 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4e2777-9283-407d-a3d7-e9dca87f8c41 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:45.289890 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4e2777-9283-407d-a3d7-e9dca87f8c41 None admin] PATH_INFO: `/v3/users/2fb4d4a4ce11400dbfcb8b0f36a5b97c` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:45.299164 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:45.305280 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4e2777-9283-407d-a3d7-e9dca87f8c41 None admin] RBAC: Authorizing `identity:get_user(user_id=2fb4d4a4ce11400dbfcb8b0f36a5b97c)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:45.306746 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4e2777-9283-407d-a3d7-e9dca87f8c41 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:45.308418 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 97/188] 10.222.0.89 () {56 vars in 1177 bytes} [Thu Jul 28 17:30:45 2022] GET /identity/v3/users/2fb4d4a4ce11400dbfcb8b0f36a5b97c => generated 283 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:30:45.313730 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4e2777-9283-407d-a3d7-e9dca87f8c41 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:45.332295 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 92/189] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:30:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:30:45.333137 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6df198-3247-4545-beb8-68cc6e95474e None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:45.429224 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4e2777-9283-407d-a3d7-e9dca87f8c41 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:45.442164 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6df198-3247-4545-beb8-68cc6e95474e None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:45.454388 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:45.465986 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376969ec-f49a-4277-86cb-7d6926caba70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:45.466628 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376969ec-f49a-4277-86cb-7d6926caba70 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:45.466906 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376969ec-f49a-4277-86cb-7d6926caba70 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:45.467177 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376969ec-f49a-4277-86cb-7d6926caba70 None admin] PATH_INFO: `/v3/projects/38d2f8adcb494ba8b566e63d96beceda` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:45.468569 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376969ec-f49a-4277-86cb-7d6926caba70 None admin] RBAC: Authorizing `identity:get_project(project_id=38d2f8adcb494ba8b566e63d96beceda)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:45.470057 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376969ec-f49a-4277-86cb-7d6926caba70 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:45.471729 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 98/190] 10.222.0.89 () {56 vars in 1186 bytes} [Thu Jul 28 17:30:45 2022] GET /identity/v3/projects/38d2f8adcb494ba8b566e63d96beceda => generated 312 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:30:45.472494 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376969ec-f49a-4277-86cb-7d6926caba70 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:45.475758 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7dcca8-c7f8-4484-afde-4d72d56d470c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:45.476787 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7dcca8-c7f8-4484-afde-4d72d56d470c None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:45.477072 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7dcca8-c7f8-4484-afde-4d72d56d470c None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:45.477376 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7dcca8-c7f8-4484-afde-4d72d56d470c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:45.488750 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7dcca8-c7f8-4484-afde-4d72d56d470c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:30:45.493715 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-9f7dcca8-c7f8-4484-afde-4d72d56d470c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 17:30:45.493715 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:30:45.493715 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:30:45.493715 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:30:45.493715 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:30:45.493715 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:30:45.493715 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:30:45.493715 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:30:45.493715 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:30:45.493715 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:30:45.493715 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:30:45.493715 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:30:45.493715 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:30:45.493715 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:30:45.493715 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:30:45.493715 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:30:45.494970 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:30:45.494970 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:30:45.494970 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:30:45.494970 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:30:45.494970 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:30:45.494970 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:30:45.494970 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:30:45.494970 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:30:45.494970 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:30:45.494970 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:30:45.494970 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:30:45.494970 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:30:45.494970 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:30:45.494970 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:30:45.494970 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:30:45.494970 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:30:45.495736 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:30:45.495736 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:30:45.495736 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:30:45.495736 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:30:45.495736 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:30:45.495736 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:30:45.495736 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:30:45.495736 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:30:45.495736 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:30:45.495736 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:30:45.495736 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 17:30:45.495736 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:30:45.495736 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 93/191] 10.222.0.89 () {56 vars in 1114 bytes} [Thu Jul 28 17:30:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:30:45.497374 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7dcca8-c7f8-4484-afde-4d72d56d470c None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:45.529169 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:45.571997 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376969ec-f49a-4277-86cb-7d6926caba70 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:45.600845 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7dcca8-c7f8-4484-afde-4d72d56d470c None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:45.605253 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9db767-ce55-410b-92fa-3d78e04940f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:45.605867 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9db767-ce55-410b-92fa-3d78e04940f6 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:45.606147 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9db767-ce55-410b-92fa-3d78e04940f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:45.606418 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9db767-ce55-410b-92fa-3d78e04940f6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:45.607204 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9db767-ce55-410b-92fa-3d78e04940f6 None admin] RBAC: Adding query filter params (role.id=895fd016dffb41e7b0205c9a55a4e0f8, scope.project.id=26339f3896c14d148dd3af450ec9d25f, user.id=7eafa4211ada43a0b09036090ff902f7) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:30:45.607516 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9db767-ce55-410b-92fa-3d78e04940f6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:45.609499 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9db767-ce55-410b-92fa-3d78e04940f6 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:45.615159 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 99/192] 10.222.0.89 () {56 vars in 1374 bytes} [Thu Jul 28 17:30:45 2022] GET /identity/v3/role_assignments?user.id=7eafa4211ada43a0b09036090ff902f7&scope.project.id=26339f3896c14d148dd3af450ec9d25f&role.id=895fd016dffb41e7b0205c9a55a4e0f8 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:30:45.616031 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9db767-ce55-410b-92fa-3d78e04940f6 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:45.639058 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bbe583-6d1c-4181-bc01-f3799a7d4bea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:45.640028 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bbe583-6d1c-4181-bc01-f3799a7d4bea None admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:45.640641 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bbe583-6d1c-4181-bc01-f3799a7d4bea None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:45.640782 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bbe583-6d1c-4181-bc01-f3799a7d4bea None admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:45.641461 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bbe583-6d1c-4181-bc01-f3799a7d4bea None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:45.642975 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bbe583-6d1c-4181-bc01-f3799a7d4bea None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:45.685234 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 94/193] 10.222.0.89 () {60 vars in 1132 bytes} [Thu Jul 28 17:30:45 2022] POST /identity/v3/users => generated 297 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:30:45.685234 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bbe583-6d1c-4181-bc01-f3799a7d4bea None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:45.702118 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:45.777602 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9db767-ce55-410b-92fa-3d78e04940f6 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:45.820317 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:45.835815 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bae6c0-5171-4650-ab7d-f0d1e1707855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:45.835815 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bae6c0-5171-4650-ab7d-f0d1e1707855 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:45.836109 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bae6c0-5171-4650-ab7d-f0d1e1707855 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:45.836759 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bae6c0-5171-4650-ab7d-f0d1e1707855 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:45.839629 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bbe583-6d1c-4181-bc01-f3799a7d4bea None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:45.882790 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa54f52-8977-49b0-a2b1-2ae5e8d7b52c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:45.883777 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa54f52-8977-49b0-a2b1-2ae5e8d7b52c None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:45.884198 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa54f52-8977-49b0-a2b1-2ae5e8d7b52c None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:45.884566 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa54f52-8977-49b0-a2b1-2ae5e8d7b52c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:45.898225 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa54f52-8977-49b0-a2b1-2ae5e8d7b52c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:30:45.903487 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-6aa54f52-8977-49b0-a2b1-2ae5e8d7b52c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 17:30:45.903487 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:30:45.903487 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:30:45.903487 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:30:45.903487 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:30:45.903487 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:30:45.903487 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:30:45.903487 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:30:45.903487 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:30:45.903487 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:30:45.903487 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:30:45.903487 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:30:45.903487 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:30:45.903487 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:30:45.903487 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:30:45.903487 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:30:45.904376 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:30:45.904376 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:30:45.904376 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:30:45.904376 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:30:45.904376 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:30:45.904376 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:30:45.904376 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:30:45.904376 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:30:45.904376 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:30:45.904376 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:30:45.904376 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:30:45.904376 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:30:45.904376 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:30:45.904376 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:30:45.904376 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:30:45.904376 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:30:45.906824 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:30:45.906824 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:30:45.906824 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:30:45.906824 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:30:45.906824 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:30:45.906824 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:30:45.906824 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:30:45.906824 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:30:45.906824 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:30:45.906824 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:30:45.906824 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 17:30:45.906824 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:30:45.906824 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 95/194] 10.222.0.89 () {56 vars in 1099 bytes} [Thu Jul 28 17:30:45 2022] GET /identity/v3/roles/reader => generated 84 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:30:45.906824 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa54f52-8977-49b0-a2b1-2ae5e8d7b52c None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:45.933213 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bae6c0-5171-4650-ab7d-f0d1e1707855 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:45.934733 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bae6c0-5171-4650-ab7d-f0d1e1707855 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:46.012891 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:46.039350 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa54f52-8977-49b0-a2b1-2ae5e8d7b52c None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:46.068038 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 100/195] 10.222.0.89 () {58 vars in 1282 bytes} [Thu Jul 28 17:30:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 17:30:46.069493 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bae6c0-5171-4650-ab7d-f0d1e1707855 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:46.084124 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed6e5e7-5e03-4075-a94d-4230388e3cd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:46.085128 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed6e5e7-5e03-4075-a94d-4230388e3cd6 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:46.085689 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed6e5e7-5e03-4075-a94d-4230388e3cd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:46.086143 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed6e5e7-5e03-4075-a94d-4230388e3cd6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:46.087239 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed6e5e7-5e03-4075-a94d-4230388e3cd6 None admin] RBAC: Adding query filter params (role.id=11f8731756fe4caca13efa61b71b11c5, scope.project.id=38d2f8adcb494ba8b566e63d96beceda, user.id=7eafa4211ada43a0b09036090ff902f7) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:30:46.087730 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed6e5e7-5e03-4075-a94d-4230388e3cd6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:46.089527 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed6e5e7-5e03-4075-a94d-4230388e3cd6 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:46.098064 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 96/196] 10.222.0.89 () {56 vars in 1374 bytes} [Thu Jul 28 17:30:45 2022] GET /identity/v3/role_assignments?user.id=7eafa4211ada43a0b09036090ff902f7&scope.project.id=38d2f8adcb494ba8b566e63d96beceda&role.id=11f8731756fe4caca13efa61b71b11c5 => generated 608 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:30:46.099285 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed6e5e7-5e03-4075-a94d-4230388e3cd6 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:46.118328 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:46.232645 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed6e5e7-5e03-4075-a94d-4230388e3cd6 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:46.236976 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bae6c0-5171-4650-ab7d-f0d1e1707855 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:46.272692 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18fd966-75f5-4f46-bdef-ba37fc3cd0e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:46.273364 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18fd966-75f5-4f46-bdef-ba37fc3cd0e2 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:46.273691 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18fd966-75f5-4f46-bdef-ba37fc3cd0e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:46.273966 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18fd966-75f5-4f46-bdef-ba37fc3cd0e2 None admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:46.274693 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18fd966-75f5-4f46-bdef-ba37fc3cd0e2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:30:46.274994 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18fd966-75f5-4f46-bdef-ba37fc3cd0e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:46.276390 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18fd966-75f5-4f46-bdef-ba37fc3cd0e2 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:46.286897 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 97/197] 10.222.0.89 () {56 vars in 1101 bytes} [Thu Jul 28 17:30:46 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:30:46.287586 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18fd966-75f5-4f46-bdef-ba37fc3cd0e2 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:46.332134 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8070040f-72c9-4b88-a508-a72e8bc317a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:46.332684 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8070040f-72c9-4b88-a508-a72e8bc317a9 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:46.332967 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8070040f-72c9-4b88-a508-a72e8bc317a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:46.333277 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8070040f-72c9-4b88-a508-a72e8bc317a9 None admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:46.334073 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8070040f-72c9-4b88-a508-a72e8bc317a9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:30:46.334378 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8070040f-72c9-4b88-a508-a72e8bc317a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:46.335825 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8070040f-72c9-4b88-a508-a72e8bc317a9 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:46.346564 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 101/198] 10.222.0.89 () {56 vars in 1111 bytes} [Thu Jul 28 17:30:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:30:46.347680 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8070040f-72c9-4b88-a508-a72e8bc317a9 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:46.377301 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:46.404662 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18fd966-75f5-4f46-bdef-ba37fc3cd0e2 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:46.427340 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:46.441921 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23641cfc-4d80-4fdf-8a09-188eaf39d5a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:46.442575 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23641cfc-4d80-4fdf-8a09-188eaf39d5a8 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:46.442874 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23641cfc-4d80-4fdf-8a09-188eaf39d5a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:46.443126 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23641cfc-4d80-4fdf-8a09-188eaf39d5a8 None admin] PATH_INFO: `/v3/projects/16690c26262a497d9b62ac127ed8bf2e` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:46.444385 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23641cfc-4d80-4fdf-8a09-188eaf39d5a8 None admin] RBAC: Authorizing `identity:get_project(project_id=16690c26262a497d9b62ac127ed8bf2e)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:46.445861 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23641cfc-4d80-4fdf-8a09-188eaf39d5a8 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:46.447707 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 98/199] 10.222.0.89 () {56 vars in 1186 bytes} [Thu Jul 28 17:30:46 2022] GET /identity/v3/projects/16690c26262a497d9b62ac127ed8bf2e => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:30:46.448298 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23641cfc-4d80-4fdf-8a09-188eaf39d5a8 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:46.472498 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8070040f-72c9-4b88-a508-a72e8bc317a9 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:46.512020 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee99f8a1-e2c3-4f5f-8052-e201194f9b4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:46.512683 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee99f8a1-e2c3-4f5f-8052-e201194f9b4d None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:46.512963 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee99f8a1-e2c3-4f5f-8052-e201194f9b4d None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:46.514294 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee99f8a1-e2c3-4f5f-8052-e201194f9b4d None admin] PATH_INFO: `/v3/users/5c88f62bf45d45afade134bc5ff1e000` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:46.516858 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee99f8a1-e2c3-4f5f-8052-e201194f9b4d None admin] RBAC: Authorizing `identity:get_user(user_id=5c88f62bf45d45afade134bc5ff1e000)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:46.518359 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee99f8a1-e2c3-4f5f-8052-e201194f9b4d None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:46.520314 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 102/200] 10.222.0.89 () {56 vars in 1177 bytes} [Thu Jul 28 17:30:46 2022] GET /identity/v3/users/5c88f62bf45d45afade134bc5ff1e000 => generated 289 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:30:46.521148 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee99f8a1-e2c3-4f5f-8052-e201194f9b4d None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:46.567410 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23641cfc-4d80-4fdf-8a09-188eaf39d5a8 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:46.613363 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314e867c-95f5-49d3-80b4-c946f7b7db35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:46.613935 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314e867c-95f5-49d3-80b4-c946f7b7db35 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:46.614216 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314e867c-95f5-49d3-80b4-c946f7b7db35 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:46.614765 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314e867c-95f5-49d3-80b4-c946f7b7db35 None admin] PATH_INFO: `/v3/users/2fb4d4a4ce11400dbfcb8b0f36a5b97c` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:46.617436 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314e867c-95f5-49d3-80b4-c946f7b7db35 None admin] RBAC: Authorizing `identity:get_user(user_id=2fb4d4a4ce11400dbfcb8b0f36a5b97c)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:46.619160 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314e867c-95f5-49d3-80b4-c946f7b7db35 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:46.621794 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 99/201] 10.222.0.89 () {56 vars in 1177 bytes} [Thu Jul 28 17:30:46 2022] GET /identity/v3/users/2fb4d4a4ce11400dbfcb8b0f36a5b97c => generated 283 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:30:46.622843 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314e867c-95f5-49d3-80b4-c946f7b7db35 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:46.628915 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:46.650589 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee99f8a1-e2c3-4f5f-8052-e201194f9b4d None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:46.661534 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:46.695340 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dfe4f8-9db4-4e6f-ae22-eb15c25e97be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:46.696232 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dfe4f8-9db4-4e6f-ae22-eb15c25e97be None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:46.696591 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dfe4f8-9db4-4e6f-ae22-eb15c25e97be None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:46.696939 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dfe4f8-9db4-4e6f-ae22-eb15c25e97be None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:46.698146 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dfe4f8-9db4-4e6f-ae22-eb15c25e97be None admin] RBAC: Adding query filter params (role.id=73e457231a824ea98c28d22610be640c, scope.project.id=16690c26262a497d9b62ac127ed8bf2e, user.id=2fb4d4a4ce11400dbfcb8b0f36a5b97c) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:30:46.698866 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dfe4f8-9db4-4e6f-ae22-eb15c25e97be None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:46.700951 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dfe4f8-9db4-4e6f-ae22-eb15c25e97be None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:46.708913 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 103/202] 10.222.0.89 () {56 vars in 1374 bytes} [Thu Jul 28 17:30:46 2022] GET /identity/v3/role_assignments?user.id=2fb4d4a4ce11400dbfcb8b0f36a5b97c&scope.project.id=16690c26262a497d9b62ac127ed8bf2e&role.id=73e457231a824ea98c28d22610be640c => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:30:46.714005 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dfe4f8-9db4-4e6f-ae22-eb15c25e97be None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:46.793843 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314e867c-95f5-49d3-80b4-c946f7b7db35 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:46.871859 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a76db4-8eb5-4b91-a315-43f6f33ded72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:46.872880 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a76db4-8eb5-4b91-a315-43f6f33ded72 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:46.873574 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a76db4-8eb5-4b91-a315-43f6f33ded72 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:46.874789 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a76db4-8eb5-4b91-a315-43f6f33ded72 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:46.937783 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dfe4f8-9db4-4e6f-ae22-eb15c25e97be None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:46.987955 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a76db4-8eb5-4b91-a315-43f6f33ded72 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:46.990351 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a76db4-8eb5-4b91-a315-43f6f33ded72 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:46.999761 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8233f01-3d2b-4110-ad92-7be55dd03d8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:47.000710 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8233f01-3d2b-4110-ad92-7be55dd03d8b None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:47.001215 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8233f01-3d2b-4110-ad92-7be55dd03d8b None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:47.001710 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8233f01-3d2b-4110-ad92-7be55dd03d8b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:47.011730 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:47.041519 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:47.102881 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8233f01-3d2b-4110-ad92-7be55dd03d8b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:47.104623 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8233f01-3d2b-4110-ad92-7be55dd03d8b None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:47.218700 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 100/203] 10.222.0.89 () {58 vars in 1282 bytes} [Thu Jul 28 17:30:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 596 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 17:30:47.222135 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 104/204] 10.222.0.89 () {58 vars in 1282 bytes} [Thu Jul 28 17:30:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 513 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 17:30:47.228206 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a76db4-8eb5-4b91-a315-43f6f33ded72 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:47.229551 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8233f01-3d2b-4110-ad92-7be55dd03d8b None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:47.331630 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a76db4-8eb5-4b91-a315-43f6f33ded72 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:47.353510 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:47.367637 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a7ae58-94cc-42ce-b81b-3facffa5383b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:47.368424 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a7ae58-94cc-42ce-b81b-3facffa5383b None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:47.368715 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a7ae58-94cc-42ce-b81b-3facffa5383b None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:47.369275 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a7ae58-94cc-42ce-b81b-3facffa5383b None admin] PATH_INFO: `/v3/projects/26339f3896c14d148dd3af450ec9d25f` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:47.370467 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a7ae58-94cc-42ce-b81b-3facffa5383b None admin] RBAC: Authorizing `identity:get_project(project_id=26339f3896c14d148dd3af450ec9d25f)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:47.371908 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a7ae58-94cc-42ce-b81b-3facffa5383b None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:47.373647 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 101/205] 10.222.0.89 () {56 vars in 1186 bytes} [Thu Jul 28 17:30:47 2022] GET /identity/v3/projects/26339f3896c14d148dd3af450ec9d25f => generated 298 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:30:47.375146 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:47.379751 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a7ae58-94cc-42ce-b81b-3facffa5383b None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:47.482826 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a7ae58-94cc-42ce-b81b-3facffa5383b None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:47.500940 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8233f01-3d2b-4110-ad92-7be55dd03d8b None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:47.540989 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72605250-477c-4de5-b46f-ceeb4b55f99c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:47.541645 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72605250-477c-4de5-b46f-ceeb4b55f99c None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:47.541926 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72605250-477c-4de5-b46f-ceeb4b55f99c None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:47.542199 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72605250-477c-4de5-b46f-ceeb4b55f99c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:47.543001 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72605250-477c-4de5-b46f-ceeb4b55f99c None admin] RBAC: Adding query filter params (role.id=466df921c2db4456b37637c23382d92a, scope.project.id=26339f3896c14d148dd3af450ec9d25f, user.id=5c88f62bf45d45afade134bc5ff1e000) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:30:47.543315 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72605250-477c-4de5-b46f-ceeb4b55f99c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:47.544753 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72605250-477c-4de5-b46f-ceeb4b55f99c None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:47.553266 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d7172d-5c6a-4a31-b52a-2c057ca96734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:47.553915 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d7172d-5c6a-4a31-b52a-2c057ca96734 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:47.554191 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d7172d-5c6a-4a31-b52a-2c057ca96734 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:47.554461 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d7172d-5c6a-4a31-b52a-2c057ca96734 None admin] PATH_INFO: `/v3/projects/16690c26262a497d9b62ac127ed8bf2e` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:47.555972 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d7172d-5c6a-4a31-b52a-2c057ca96734 None admin] RBAC: Authorizing `identity:get_project(project_id=16690c26262a497d9b62ac127ed8bf2e)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:47.557431 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d7172d-5c6a-4a31-b52a-2c057ca96734 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:47.566902 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 102/207] 10.222.0.89 () {56 vars in 1374 bytes} [Thu Jul 28 17:30:47 2022] GET /identity/v3/role_assignments?user.id=5c88f62bf45d45afade134bc5ff1e000&scope.project.id=26339f3896c14d148dd3af450ec9d25f&role.id=466df921c2db4456b37637c23382d92a => generated 608 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:30:47.567087 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 105/207] 10.222.0.89 () {56 vars in 1186 bytes} [Thu Jul 28 17:30:47 2022] GET /identity/v3/projects/16690c26262a497d9b62ac127ed8bf2e => generated 302 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:30:47.569467 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72605250-477c-4de5-b46f-ceeb4b55f99c None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:47.639801 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:47.728289 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72605250-477c-4de5-b46f-ceeb4b55f99c None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:47.772978 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9eb91a-62c0-4593-a9db-6b9956f8cc6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:47.773739 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9eb91a-62c0-4593-a9db-6b9956f8cc6f None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:47.774026 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9eb91a-62c0-4593-a9db-6b9956f8cc6f None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:47.774308 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9eb91a-62c0-4593-a9db-6b9956f8cc6f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:47.775113 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9eb91a-62c0-4593-a9db-6b9956f8cc6f None admin] RBAC: Adding query filter params (role.id=895fd016dffb41e7b0205c9a55a4e0f8, scope.project.id=16690c26262a497d9b62ac127ed8bf2e, user.id=2fb4d4a4ce11400dbfcb8b0f36a5b97c) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:30:47.775422 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9eb91a-62c0-4593-a9db-6b9956f8cc6f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:47.776866 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9eb91a-62c0-4593-a9db-6b9956f8cc6f None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:47.783727 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 103/208] 10.222.0.89 () {56 vars in 1374 bytes} [Thu Jul 28 17:30:47 2022] GET /identity/v3/role_assignments?user.id=2fb4d4a4ce11400dbfcb8b0f36a5b97c&scope.project.id=16690c26262a497d9b62ac127ed8bf2e&role.id=895fd016dffb41e7b0205c9a55a4e0f8 => generated 259 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:30:48.209901 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b990ec-6df3-4add-830f-f69efedf5ee4 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:48.210429 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b990ec-6df3-4add-830f-f69efedf5ee4 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:48.210869 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b990ec-6df3-4add-830f-f69efedf5ee4 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:48.212608 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 106/209] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:30:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:30:48.218375 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702b369e-ed6a-48c6-8fd1-29be6193347a None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:48.218948 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702b369e-ed6a-48c6-8fd1-29be6193347a None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:48.219846 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702b369e-ed6a-48c6-8fd1-29be6193347a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:48.238970 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-702b369e-ed6a-48c6-8fd1-29be6193347a None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:30:48.240502 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-702b369e-ed6a-48c6-8fd1-29be6193347a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:30:48.304384 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293d8723-0c94-4547-9461-305a2697436e None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:48.304987 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293d8723-0c94-4547-9461-305a2697436e None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:48.305481 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293d8723-0c94-4547-9461-305a2697436e None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:48.307258 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 107/210] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:30:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:30:48.313125 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afa8014-f053-4816-849d-8afc08d00bfa None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:48.313729 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afa8014-f053-4816-849d-8afc08d00bfa None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:48.314223 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afa8014-f053-4816-849d-8afc08d00bfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:48.315978 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 104/211] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:30:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:30:48.337423 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-5afa8014-f053-4816-849d-8afc08d00bfa None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:30:48.338868 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-5afa8014-f053-4816-849d-8afc08d00bfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:30:48.341748 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702b369e-ed6a-48c6-8fd1-29be6193347a None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:48.380504 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:48.437127 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 108/212] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:30:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:30:48.455212 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702b369e-ed6a-48c6-8fd1-29be6193347a None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:48.460671 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afa8014-f053-4816-849d-8afc08d00bfa None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:48.491768 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e150689-0ad6-42c1-8188-1887c8e6447d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:48.492426 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e150689-0ad6-42c1-8188-1887c8e6447d None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:48.492742 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e150689-0ad6-42c1-8188-1887c8e6447d None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:48.493016 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e150689-0ad6-42c1-8188-1887c8e6447d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:48.494454 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e150689-0ad6-42c1-8188-1887c8e6447d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:48.495905 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e150689-0ad6-42c1-8188-1887c8e6447d None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:48.497844 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 105/213] 10.222.0.89 () {56 vars in 1108 bytes} [Thu Jul 28 17:30:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:30:48.503885 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e150689-0ad6-42c1-8188-1887c8e6447d None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:48.512964 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:48.569473 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afa8014-f053-4816-849d-8afc08d00bfa None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:48.605373 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fcc5c6-a5e2-4bcb-a2d0-76ecbe2af3b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:48.606018 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fcc5c6-a5e2-4bcb-a2d0-76ecbe2af3b6 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:48.606295 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fcc5c6-a5e2-4bcb-a2d0-76ecbe2af3b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:48.606566 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fcc5c6-a5e2-4bcb-a2d0-76ecbe2af3b6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:48.609966 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e150689-0ad6-42c1-8188-1887c8e6447d None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:48.612482 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:48.618292 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fcc5c6-a5e2-4bcb-a2d0-76ecbe2af3b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:30:48.623587 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-34fcc5c6-a5e2-4bcb-a2d0-76ecbe2af3b6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 17:30:48.623587 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:30:48.623587 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:30:48.623587 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:30:48.623587 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:30:48.623587 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:30:48.623587 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:30:48.623587 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:30:48.623587 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:30:48.623587 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:30:48.623587 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:30:48.623587 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:30:48.623587 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:30:48.623587 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:30:48.623587 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:30:48.623587 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:30:48.625018 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:30:48.625018 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:30:48.625018 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:30:48.625018 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:30:48.625018 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:30:48.625018 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:30:48.625018 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:30:48.625018 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:30:48.625018 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:30:48.625018 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:30:48.625018 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:30:48.625018 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( Jul 28 17:30:48.625018 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:30:48.625018 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:30:48.625018 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:30:48.625018 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:30:48.627277 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:30:48.627277 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:30:48.627277 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:30:48.627277 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:30:48.627277 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:30:48.627277 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:30:48.627277 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:30:48.627277 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:30:48.627277 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:30:48.627277 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:30:48.627277 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 17:30:48.627277 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:30:48.627277 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 109/214] 10.222.0.89 () {56 vars in 1099 bytes} [Thu Jul 28 17:30:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:30:48.633181 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fcc5c6-a5e2-4bcb-a2d0-76ecbe2af3b6 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:48.647879 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f162401-d3b1-429c-96c7-4c02b1ea717a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:48.648509 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f162401-d3b1-429c-96c7-4c02b1ea717a None admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:48.648785 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f162401-d3b1-429c-96c7-4c02b1ea717a None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:48.649054 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f162401-d3b1-429c-96c7-4c02b1ea717a None admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:48.649783 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f162401-d3b1-429c-96c7-4c02b1ea717a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:48.651228 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f162401-d3b1-429c-96c7-4c02b1ea717a None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:48.681279 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 106/215] 10.222.0.89 () {60 vars in 1132 bytes} [Thu Jul 28 17:30:48 2022] POST /identity/v3/users => generated 297 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:30:48.752728 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fcc5c6-a5e2-4bcb-a2d0-76ecbe2af3b6 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:48.763960 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:48.793328 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a483e62-ff3b-45b4-a7ff-3a6e1c2018b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:48.793963 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a483e62-ff3b-45b4-a7ff-3a6e1c2018b6 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:48.794242 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a483e62-ff3b-45b4-a7ff-3a6e1c2018b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:48.794513 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a483e62-ff3b-45b4-a7ff-3a6e1c2018b6 None admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:48.795248 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a483e62-ff3b-45b4-a7ff-3a6e1c2018b6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:30:48.795549 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a483e62-ff3b-45b4-a7ff-3a6e1c2018b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:48.797838 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a483e62-ff3b-45b4-a7ff-3a6e1c2018b6 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:48.807971 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 110/216] 10.222.0.89 () {56 vars in 1101 bytes} [Thu Jul 28 17:30:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:30:48.816254 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f162401-d3b1-429c-96c7-4c02b1ea717a None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:48.926129 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f162401-d3b1-429c-96c7-4c02b1ea717a None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:48.937758 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceef6b4-47b0-4f2b-b356-0efa2ab1fe0f None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:48.938046 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceef6b4-47b0-4f2b-b356-0efa2ab1fe0f None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:48.938317 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceef6b4-47b0-4f2b-b356-0efa2ab1fe0f None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:48.940021 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 111/217] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:30:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:30:48.946538 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf78734-8637-4c30-96f1-86791d804ada None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:48.946828 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf78734-8637-4c30-96f1-86791d804ada None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:48.947105 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf78734-8637-4c30-96f1-86791d804ada None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:48.963942 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eac11c8-33c2-4587-9b36-71fe73a12b35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:48.964602 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eac11c8-33c2-4587-9b36-71fe73a12b35 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:48.965771 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eac11c8-33c2-4587-9b36-71fe73a12b35 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:48.966073 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eac11c8-33c2-4587-9b36-71fe73a12b35 None admin] PATH_INFO: `/v3/users/d763f6f23f304e44a2c1ac9028371ae8` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:48.978212 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-7bf78734-8637-4c30-96f1-86791d804ada None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:30:48.979456 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-7bf78734-8637-4c30-96f1-86791d804ada None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:30:48.983262 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eac11c8-33c2-4587-9b36-71fe73a12b35 None admin] RBAC: Authorizing `identity:get_user(user_id=d763f6f23f304e44a2c1ac9028371ae8)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:48.986071 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eac11c8-33c2-4587-9b36-71fe73a12b35 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:48.988059 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 107/218] 10.222.0.89 () {56 vars in 1177 bytes} [Thu Jul 28 17:30:48 2022] GET /identity/v3/users/d763f6f23f304e44a2c1ac9028371ae8 => generated 297 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:30:48.995078 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eac11c8-33c2-4587-9b36-71fe73a12b35 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:49.019907 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:49.072050 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 112/219] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:30:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:30:49.099490 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf78734-8637-4c30-96f1-86791d804ada None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:49.131721 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eac11c8-33c2-4587-9b36-71fe73a12b35 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:49.178954 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209d5dbb-f5a5-4c6c-8e0e-40814e10f9c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:49.179586 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209d5dbb-f5a5-4c6c-8e0e-40814e10f9c2 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:49.179868 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209d5dbb-f5a5-4c6c-8e0e-40814e10f9c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:49.180145 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209d5dbb-f5a5-4c6c-8e0e-40814e10f9c2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:49.220924 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:49.228112 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf78734-8637-4c30-96f1-86791d804ada None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:49.268456 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67890a1-da6d-43d0-8f45-701a2afdc8a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:49.269105 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67890a1-da6d-43d0-8f45-701a2afdc8a6 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:49.269422 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67890a1-da6d-43d0-8f45-701a2afdc8a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:49.269700 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67890a1-da6d-43d0-8f45-701a2afdc8a6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:49.281119 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67890a1-da6d-43d0-8f45-701a2afdc8a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:30:49.282588 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209d5dbb-f5a5-4c6c-8e0e-40814e10f9c2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:49.284592 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209d5dbb-f5a5-4c6c-8e0e-40814e10f9c2 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:49.286097 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-e67890a1-da6d-43d0-8f45-701a2afdc8a6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:30:49.286097 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:30:49.286097 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:30:49.286097 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:30:49.286097 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:30:49.286097 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:30:49.286097 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:30:49.286097 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:30:49.286097 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:30:49.286097 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:30:49.286097 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:30:49.286097 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:30:49.286097 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:30:49.286097 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:30:49.286097 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:30:49.286097 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:30:49.286965 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:30:49.286965 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:30:49.286965 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:30:49.286965 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:30:49.286965 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:30:49.286965 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:30:49.286965 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:30:49.286965 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:30:49.286965 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:30:49.286965 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:30:49.286965 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:30:49.286965 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( Jul 28 17:30:49.286965 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:30:49.286965 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:30:49.286965 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:30:49.286965 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:30:49.287768 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:30:49.287768 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:30:49.287768 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:30:49.287768 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:30:49.287768 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:30:49.287768 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:30:49.287768 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:30:49.287768 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:30:49.287768 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:30:49.287768 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:30:49.287768 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:30:49.287768 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:30:49.287768 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 113/220] 10.222.0.89 () {56 vars in 1096 bytes} [Thu Jul 28 17:30:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:30:49.294829 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67890a1-da6d-43d0-8f45-701a2afdc8a6 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:49.310518 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:49.424808 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 108/221] 10.222.0.89 () {58 vars in 1282 bytes} [Thu Jul 28 17:30:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 28 17:30:49.430048 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209d5dbb-f5a5-4c6c-8e0e-40814e10f9c2 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:49.477025 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67890a1-da6d-43d0-8f45-701a2afdc8a6 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:49.547178 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4188d27f-7b65-4a1f-b693-e066353270c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:49.549185 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4188d27f-7b65-4a1f-b693-e066353270c1 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:49.550772 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4188d27f-7b65-4a1f-b693-e066353270c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:49.551195 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4188d27f-7b65-4a1f-b693-e066353270c1 None admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:49.552298 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4188d27f-7b65-4a1f-b693-e066353270c1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:30:49.555186 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4188d27f-7b65-4a1f-b693-e066353270c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:49.556791 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4188d27f-7b65-4a1f-b693-e066353270c1 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:49.578794 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 114/222] 10.222.0.89 () {56 vars in 1099 bytes} [Thu Jul 28 17:30:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:30:49.585581 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4188d27f-7b65-4a1f-b693-e066353270c1 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:49.615665 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:49.628774 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209d5dbb-f5a5-4c6c-8e0e-40814e10f9c2 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:49.664676 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:49.705346 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dca2be2-3a61-4037-a30f-d040c173004d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:49.706112 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dca2be2-3a61-4037-a30f-d040c173004d None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:49.706401 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dca2be2-3a61-4037-a30f-d040c173004d None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:49.706685 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dca2be2-3a61-4037-a30f-d040c173004d None admin] PATH_INFO: `/v3/projects/16690c26262a497d9b62ac127ed8bf2e` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:49.709450 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dca2be2-3a61-4037-a30f-d040c173004d None admin] RBAC: Authorizing `identity:get_project(project_id=16690c26262a497d9b62ac127ed8bf2e)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:49.711001 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dca2be2-3a61-4037-a30f-d040c173004d None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:49.712970 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 109/223] 10.222.0.89 () {56 vars in 1186 bytes} [Thu Jul 28 17:30:49 2022] GET /identity/v3/projects/16690c26262a497d9b62ac127ed8bf2e => generated 302 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:30:49.716470 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4188d27f-7b65-4a1f-b693-e066353270c1 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:49.719774 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dca2be2-3a61-4037-a30f-d040c173004d None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:49.759360 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc7f757-0e98-49d4-9f8d-213684532253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:49.760248 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc7f757-0e98-49d4-9f8d-213684532253 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:49.760672 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc7f757-0e98-49d4-9f8d-213684532253 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:49.761087 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc7f757-0e98-49d4-9f8d-213684532253 None admin] PATH_INFO: `/v3/users/2fb4d4a4ce11400dbfcb8b0f36a5b97c` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:49.762858 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc7f757-0e98-49d4-9f8d-213684532253 None admin] RBAC: Authorizing `identity:get_user(user_id=2fb4d4a4ce11400dbfcb8b0f36a5b97c)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:49.765297 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc7f757-0e98-49d4-9f8d-213684532253 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:49.767721 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 115/224] 10.222.0.89 () {56 vars in 1177 bytes} [Thu Jul 28 17:30:49 2022] GET /identity/v3/users/2fb4d4a4ce11400dbfcb8b0f36a5b97c => generated 283 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:30:49.773687 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc7f757-0e98-49d4-9f8d-213684532253 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:49.905648 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:49.924786 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc7f757-0e98-49d4-9f8d-213684532253 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:49.968159 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dca2be2-3a61-4037-a30f-d040c173004d None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:49.997244 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dc6b74-80c8-47c2-a0d3-5b7867d95c5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:49.997895 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dc6b74-80c8-47c2-a0d3-5b7867d95c5c None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:49.998184 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dc6b74-80c8-47c2-a0d3-5b7867d95c5c None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:49.998464 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dc6b74-80c8-47c2-a0d3-5b7867d95c5c None admin] PATH_INFO: `/v3/projects/16690c26262a497d9b62ac127ed8bf2e` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:50.000451 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dc6b74-80c8-47c2-a0d3-5b7867d95c5c None admin] RBAC: Authorizing `identity:get_project(project_id=16690c26262a497d9b62ac127ed8bf2e)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:50.002279 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dc6b74-80c8-47c2-a0d3-5b7867d95c5c None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:50.004087 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 116/225] 10.222.0.89 () {56 vars in 1186 bytes} [Thu Jul 28 17:30:49 2022] GET /identity/v3/projects/16690c26262a497d9b62ac127ed8bf2e => generated 302 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:30:50.005495 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b293fd-111d-43ac-973d-2c1697412314 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:50.005782 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b293fd-111d-43ac-973d-2c1697412314 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:50.006059 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b293fd-111d-43ac-973d-2c1697412314 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:50.007283 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 117/226] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:30:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:30:50.009339 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d81a63-1174-4855-abcd-d9339fed1e2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:50.010019 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d81a63-1174-4855-abcd-d9339fed1e2e None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:50.010304 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d81a63-1174-4855-abcd-d9339fed1e2e None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:50.010580 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d81a63-1174-4855-abcd-d9339fed1e2e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:50.011375 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d81a63-1174-4855-abcd-d9339fed1e2e None admin] RBAC: Adding query filter params (role.id=11f8731756fe4caca13efa61b71b11c5, scope.project.id=16690c26262a497d9b62ac127ed8bf2e, user.id=d763f6f23f304e44a2c1ac9028371ae8) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:30:50.011684 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d81a63-1174-4855-abcd-d9339fed1e2e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:50.012204 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b293fd-111d-43ac-973d-2c1697412314 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:50.013262 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d81a63-1174-4855-abcd-d9339fed1e2e None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:50.020740 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 110/227] 10.222.0.89 () {56 vars in 1374 bytes} [Thu Jul 28 17:30:49 2022] GET /identity/v3/role_assignments?user.id=d763f6f23f304e44a2c1ac9028371ae8&scope.project.id=16690c26262a497d9b62ac127ed8bf2e&role.id=11f8731756fe4caca13efa61b71b11c5 => generated 259 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:30:50.021974 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f925480a-9948-4f6f-acc5-879afeb94056 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:50.022732 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f925480a-9948-4f6f-acc5-879afeb94056 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:50.023068 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f925480a-9948-4f6f-acc5-879afeb94056 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:50.054410 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-f925480a-9948-4f6f-acc5-879afeb94056 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:30:50.056703 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-f925480a-9948-4f6f-acc5-879afeb94056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:30:50.111948 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:50.127545 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b293fd-111d-43ac-973d-2c1697412314 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:50.137911 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 111/228] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:30:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:30:50.170255 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f925480a-9948-4f6f-acc5-879afeb94056 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:50.192847 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac95959-4545-4b98-b707-5cc104aa1fa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:50.193641 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac95959-4545-4b98-b707-5cc104aa1fa3 None admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:50.194091 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac95959-4545-4b98-b707-5cc104aa1fa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:50.194598 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac95959-4545-4b98-b707-5cc104aa1fa3 None admin] PATH_INFO: `/v3/projects/16690c26262a497d9b62ac127ed8bf2e/users/2fb4d4a4ce11400dbfcb8b0f36a5b97c/roles/73e457231a824ea98c28d22610be640c` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:50.200006 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac95959-4545-4b98-b707-5cc104aa1fa3 None admin] RBAC: Authorizing `identity:create_grant(project_id=16690c26262a497d9b62ac127ed8bf2e, user_id=2fb4d4a4ce11400dbfcb8b0f36a5b97c, role_id=73e457231a824ea98c28d22610be640c)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:50.202615 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac95959-4545-4b98-b707-5cc104aa1fa3 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:50.229978 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 118/229] 10.222.0.89 () {58 vars in 1439 bytes} [Thu Jul 28 17:30:50 2022] PUT /identity/v3/projects/16690c26262a497d9b62ac127ed8bf2e/users/2fb4d4a4ce11400dbfcb8b0f36a5b97c/roles/73e457231a824ea98c28d22610be640c => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:30:50.324646 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f925480a-9948-4f6f-acc5-879afeb94056 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:50.365512 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67b65f9-2074-4471-981a-165a32fcc7ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:50.366166 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67b65f9-2074-4471-981a-165a32fcc7ff None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:50.366454 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67b65f9-2074-4471-981a-165a32fcc7ff None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:50.366728 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67b65f9-2074-4471-981a-165a32fcc7ff None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:50.381476 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67b65f9-2074-4471-981a-165a32fcc7ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:30:50.386655 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-f67b65f9-2074-4471-981a-165a32fcc7ff None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 17:30:50.386655 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:30:50.386655 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:30:50.386655 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:30:50.386655 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:30:50.386655 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:30:50.386655 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:30:50.386655 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:30:50.386655 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:30:50.386655 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:30:50.386655 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:30:50.386655 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:30:50.386655 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:30:50.386655 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:30:50.386655 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:30:50.386655 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:30:50.387563 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:30:50.387563 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:30:50.387563 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:30:50.387563 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:30:50.387563 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:30:50.387563 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:30:50.387563 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:30:50.387563 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:30:50.387563 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:30:50.387563 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:30:50.387563 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:30:50.387563 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:30:50.387563 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:30:50.387563 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:30:50.387563 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:30:50.387563 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:30:50.388347 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:30:50.388347 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:30:50.388347 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:30:50.388347 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:30:50.388347 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:30:50.388347 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:30:50.388347 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:30:50.388347 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:30:50.388347 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:30:50.388347 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:30:50.388347 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 17:30:50.388347 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:30:50.388347 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 112/230] 10.222.0.89 () {56 vars in 1114 bytes} [Thu Jul 28 17:30:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:30:50.394669 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac95959-4545-4b98-b707-5cc104aa1fa3 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:50.417625 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:50.528001 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac95959-4545-4b98-b707-5cc104aa1fa3 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:50.566910 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faeb40cc-2b1c-477c-8fb2-780f3c3453d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:50.567532 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeb40cc-2b1c-477c-8fb2-780f3c3453d5 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:50.567795 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeb40cc-2b1c-477c-8fb2-780f3c3453d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:50.568066 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeb40cc-2b1c-477c-8fb2-780f3c3453d5 None admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:50.568782 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faeb40cc-2b1c-477c-8fb2-780f3c3453d5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:30:50.569083 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faeb40cc-2b1c-477c-8fb2-780f3c3453d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:50.570473 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faeb40cc-2b1c-477c-8fb2-780f3c3453d5 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:50.579906 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 119/231] 10.222.0.89 () {56 vars in 1111 bytes} [Thu Jul 28 17:30:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:30:50.586236 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67b65f9-2074-4471-981a-165a32fcc7ff None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:50.650223 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:50.711140 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67b65f9-2074-4471-981a-165a32fcc7ff None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:50.748584 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f97d02-08df-4775-b511-664af757f135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:50.749474 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f97d02-08df-4775-b511-664af757f135 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:50.749698 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f97d02-08df-4775-b511-664af757f135 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:50.749972 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f97d02-08df-4775-b511-664af757f135 None admin] PATH_INFO: `/v3/users/2fb4d4a4ce11400dbfcb8b0f36a5b97c` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:50.751170 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f97d02-08df-4775-b511-664af757f135 None admin] RBAC: Authorizing `identity:get_user(user_id=2fb4d4a4ce11400dbfcb8b0f36a5b97c)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:50.752622 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f97d02-08df-4775-b511-664af757f135 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:50.754432 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 113/232] 10.222.0.89 () {56 vars in 1177 bytes} [Thu Jul 28 17:30:50 2022] GET /identity/v3/users/2fb4d4a4ce11400dbfcb8b0f36a5b97c => generated 283 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:30:50.759765 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faeb40cc-2b1c-477c-8fb2-780f3c3453d5 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:50.880324 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:50.907075 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faeb40cc-2b1c-477c-8fb2-780f3c3453d5 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:50.950804 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103bf05a-3a73-45fc-b444-f33080281373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:50.951456 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103bf05a-3a73-45fc-b444-f33080281373 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:50.951739 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103bf05a-3a73-45fc-b444-f33080281373 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:50.952563 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103bf05a-3a73-45fc-b444-f33080281373 None admin] PATH_INFO: `/v3/projects/16690c26262a497d9b62ac127ed8bf2e` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:50.954175 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103bf05a-3a73-45fc-b444-f33080281373 None admin] RBAC: Authorizing `identity:get_project(project_id=16690c26262a497d9b62ac127ed8bf2e)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:50.955636 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103bf05a-3a73-45fc-b444-f33080281373 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:50.957358 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 120/233] 10.222.0.89 () {56 vars in 1186 bytes} [Thu Jul 28 17:30:50 2022] GET /identity/v3/projects/16690c26262a497d9b62ac127ed8bf2e => generated 302 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:30:50.963135 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f97d02-08df-4775-b511-664af757f135 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:51.072830 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f97d02-08df-4775-b511-664af757f135 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:51.085519 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:51.113824 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cc98e0-2d7f-4204-8707-4e7773544abe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:51.114522 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cc98e0-2d7f-4204-8707-4e7773544abe None admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:51.114857 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cc98e0-2d7f-4204-8707-4e7773544abe None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:51.115131 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cc98e0-2d7f-4204-8707-4e7773544abe None admin] PATH_INFO: `/v3/projects/16690c26262a497d9b62ac127ed8bf2e/users/2fb4d4a4ce11400dbfcb8b0f36a5b97c/roles/895fd016dffb41e7b0205c9a55a4e0f8` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:51.123015 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cc98e0-2d7f-4204-8707-4e7773544abe None admin] RBAC: Authorizing `identity:create_grant(project_id=16690c26262a497d9b62ac127ed8bf2e, user_id=2fb4d4a4ce11400dbfcb8b0f36a5b97c, role_id=895fd016dffb41e7b0205c9a55a4e0f8)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:51.124740 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cc98e0-2d7f-4204-8707-4e7773544abe None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:51.144210 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 114/234] 10.222.0.89 () {58 vars in 1439 bytes} [Thu Jul 28 17:30:50 2022] PUT /identity/v3/projects/16690c26262a497d9b62ac127ed8bf2e/users/2fb4d4a4ce11400dbfcb8b0f36a5b97c/roles/895fd016dffb41e7b0205c9a55a4e0f8 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:30:51.443736 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0360b12-9f3b-40fc-aa10-6b94fc330649 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:51.444024 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0360b12-9f3b-40fc-aa10-6b94fc330649 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:51.444291 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0360b12-9f3b-40fc-aa10-6b94fc330649 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:51.445971 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 121/235] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:30:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:30:51.452770 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada480c9-3635-4bd0-8219-714bd648c743 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:51.452770 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada480c9-3635-4bd0-8219-714bd648c743 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:51.453534 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada480c9-3635-4bd0-8219-714bd648c743 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:51.473874 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-ada480c9-3635-4bd0-8219-714bd648c743 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:30:51.475109 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-ada480c9-3635-4bd0-8219-714bd648c743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:30:51.555936 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 115/236] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:30:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:30:51.592746 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0360b12-9f3b-40fc-aa10-6b94fc330649 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:51.739397 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0360b12-9f3b-40fc-aa10-6b94fc330649 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:51.789964 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e261ec-4b3d-41fb-90ce-d200eb1f5a80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:51.790613 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e261ec-4b3d-41fb-90ce-d200eb1f5a80 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:51.790889 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e261ec-4b3d-41fb-90ce-d200eb1f5a80 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:51.791163 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e261ec-4b3d-41fb-90ce-d200eb1f5a80 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:51.799002 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b203609-8b8e-4658-85a0-74e1517c2de0 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:51.799531 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b203609-8b8e-4658-85a0-74e1517c2de0 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:51.799951 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b203609-8b8e-4658-85a0-74e1517c2de0 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:51.802984 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 116/237] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:30:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:30:51.807481 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd9cd65-ef16-4c19-909f-30a5539464dd None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:51.808071 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd9cd65-ef16-4c19-909f-30a5539464dd None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:51.808590 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd9cd65-ef16-4c19-909f-30a5539464dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:51.809544 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e261ec-4b3d-41fb-90ce-d200eb1f5a80 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:30:51.816991 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-a4e261ec-4b3d-41fb-90ce-d200eb1f5a80 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 17:30:51.816991 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:30:51.816991 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:30:51.816991 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:30:51.816991 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:30:51.816991 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:30:51.816991 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:30:51.816991 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:30:51.816991 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:30:51.816991 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:30:51.816991 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:30:51.816991 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:30:51.816991 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:30:51.816991 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:30:51.816991 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:30:51.816991 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:30:51.818005 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:30:51.818005 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:30:51.818005 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:30:51.818005 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:30:51.818005 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:30:51.818005 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:30:51.818005 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:30:51.818005 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:30:51.818005 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:30:51.818005 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:30:51.818005 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:30:51.818005 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( Jul 28 17:30:51.818005 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:30:51.818005 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:30:51.818005 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:30:51.818005 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:30:51.819451 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:30:51.819451 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:30:51.819451 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:30:51.819451 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:30:51.819451 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:30:51.819451 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:30:51.819451 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:30:51.819451 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:30:51.819451 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:30:51.819451 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:30:51.819451 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 17:30:51.819451 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:30:51.819451 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 122/238] 10.222.0.89 () {56 vars in 1099 bytes} [Thu Jul 28 17:30:51 2022] GET /identity/v3/roles/reader => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:30:51.829195 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e261ec-4b3d-41fb-90ce-d200eb1f5a80 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:51.845054 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:51.858766 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-0cd9cd65-ef16-4c19-909f-30a5539464dd None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:30:51.860240 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-0cd9cd65-ef16-4c19-909f-30a5539464dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:30:51.969956 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e261ec-4b3d-41fb-90ce-d200eb1f5a80 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:51.990327 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 117/239] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:30:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 17:30:52.017760 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdc410d-1630-4af3-bb8a-5f6b740a2543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:52.018378 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdc410d-1630-4af3-bb8a-5f6b740a2543 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:52.018653 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdc410d-1630-4af3-bb8a-5f6b740a2543 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:52.018919 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdc410d-1630-4af3-bb8a-5f6b740a2543 None admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:52.019640 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdc410d-1630-4af3-bb8a-5f6b740a2543 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:30:52.019939 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdc410d-1630-4af3-bb8a-5f6b740a2543 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:52.021342 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdc410d-1630-4af3-bb8a-5f6b740a2543 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:52.029821 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 123/240] 10.222.0.89 () {56 vars in 1101 bytes} [Thu Jul 28 17:30:51 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:30:52.036379 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd9cd65-ef16-4c19-909f-30a5539464dd None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:52.048896 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdc410d-1630-4af3-bb8a-5f6b740a2543 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:52.084281 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:52.181369 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:52.207115 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd9cd65-ef16-4c19-909f-30a5539464dd None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:52.246348 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a236a6-ae21-43b5-b5e8-e6e5a7248de1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:52.246928 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a236a6-ae21-43b5-b5e8-e6e5a7248de1 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:52.247205 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a236a6-ae21-43b5-b5e8-e6e5a7248de1 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:52.247477 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a236a6-ae21-43b5-b5e8-e6e5a7248de1 None admin] PATH_INFO: `/v3/users/b2c2dfb020434b11af87233d93f83e0f` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:52.248773 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdc410d-1630-4af3-bb8a-5f6b740a2543 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:52.275806 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a236a6-ae21-43b5-b5e8-e6e5a7248de1 None admin] RBAC: Authorizing `identity:get_user(user_id=b2c2dfb020434b11af87233d93f83e0f)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:52.277306 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a236a6-ae21-43b5-b5e8-e6e5a7248de1 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:52.279212 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 118/241] 10.222.0.89 () {56 vars in 1177 bytes} [Thu Jul 28 17:30:52 2022] GET /identity/v3/users/b2c2dfb020434b11af87233d93f83e0f => generated 297 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:30:52.284465 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a236a6-ae21-43b5-b5e8-e6e5a7248de1 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:52.322713 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635526a3-5bd6-4987-b00a-18ae238ff841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:52.323592 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635526a3-5bd6-4987-b00a-18ae238ff841 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:52.324011 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635526a3-5bd6-4987-b00a-18ae238ff841 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:52.324924 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635526a3-5bd6-4987-b00a-18ae238ff841 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:52.328534 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635526a3-5bd6-4987-b00a-18ae238ff841 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:52.331393 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635526a3-5bd6-4987-b00a-18ae238ff841 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:52.334355 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 124/242] 10.222.0.89 () {56 vars in 1108 bytes} [Thu Jul 28 17:30:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:30:52.346058 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635526a3-5bd6-4987-b00a-18ae238ff841 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:52.413020 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:52.428767 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a236a6-ae21-43b5-b5e8-e6e5a7248de1 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:52.489507 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8131cee0-f76e-4652-a867-12bca6a35901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:52.492612 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8131cee0-f76e-4652-a867-12bca6a35901 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:52.492888 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8131cee0-f76e-4652-a867-12bca6a35901 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:52.493168 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8131cee0-f76e-4652-a867-12bca6a35901 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:52.558449 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635526a3-5bd6-4987-b00a-18ae238ff841 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:52.599346 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8131cee0-f76e-4652-a867-12bca6a35901 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:52.601260 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8131cee0-f76e-4652-a867-12bca6a35901 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:52.602880 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a997e8da-f36e-482a-9bb1-590caf554a6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:52.605142 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a997e8da-f36e-482a-9bb1-590caf554a6c None admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:52.605621 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a997e8da-f36e-482a-9bb1-590caf554a6c None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:52.606039 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a997e8da-f36e-482a-9bb1-590caf554a6c None admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:52.606876 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a997e8da-f36e-482a-9bb1-590caf554a6c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:52.608523 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a997e8da-f36e-482a-9bb1-590caf554a6c None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:52.622620 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:52.643593 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 125/243] 10.222.0.89 () {60 vars in 1132 bytes} [Thu Jul 28 17:30:52 2022] POST /identity/v3/users => generated 293 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:30:52.645201 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5922a328-72cb-4da4-acdb-142d8bfc8d3f None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:52.645634 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5922a328-72cb-4da4-acdb-142d8bfc8d3f None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:52.646056 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5922a328-72cb-4da4-acdb-142d8bfc8d3f None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:52.648909 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 126/244] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:30:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:30:52.659510 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a827503-7b24-4ad3-bfab-92b0dc76a6e5 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:52.660034 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a827503-7b24-4ad3-bfab-92b0dc76a6e5 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:52.660460 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a827503-7b24-4ad3-bfab-92b0dc76a6e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:52.696604 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-8a827503-7b24-4ad3-bfab-92b0dc76a6e5 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:30:52.698192 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-8a827503-7b24-4ad3-bfab-92b0dc76a6e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:30:52.766684 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:52.798240 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 119/245] 10.222.0.89 () {58 vars in 1282 bytes} [Thu Jul 28 17:30:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 514 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 28 17:30:52.803736 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8131cee0-f76e-4652-a867-12bca6a35901 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:52.821611 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 127/246] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:30:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:30:52.852224 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a827503-7b24-4ad3-bfab-92b0dc76a6e5 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:52.956586 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8131cee0-f76e-4652-a867-12bca6a35901 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:52.981269 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a827503-7b24-4ad3-bfab-92b0dc76a6e5 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:53.010081 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794c50a7-a9ab-45ae-af60-5bd94f0739d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:53.010982 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794c50a7-a9ab-45ae-af60-5bd94f0739d1 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:53.011263 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794c50a7-a9ab-45ae-af60-5bd94f0739d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:53.011537 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794c50a7-a9ab-45ae-af60-5bd94f0739d1 None admin] PATH_INFO: `/v3/projects/16690c26262a497d9b62ac127ed8bf2e` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:53.012712 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794c50a7-a9ab-45ae-af60-5bd94f0739d1 None admin] RBAC: Authorizing `identity:get_project(project_id=16690c26262a497d9b62ac127ed8bf2e)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:53.013204 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:53.015204 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794c50a7-a9ab-45ae-af60-5bd94f0739d1 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:53.019991 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 120/247] 10.222.0.89 () {56 vars in 1186 bytes} [Thu Jul 28 17:30:52 2022] GET /identity/v3/projects/16690c26262a497d9b62ac127ed8bf2e => generated 302 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:30:53.019991 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71cd658-0060-4fd9-a56a-bc4c4d05df2f None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:53.019991 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71cd658-0060-4fd9-a56a-bc4c4d05df2f None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:53.019991 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71cd658-0060-4fd9-a56a-bc4c4d05df2f None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:53.020817 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 121/248] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:30:53 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:30:53.028416 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71cd658-0060-4fd9-a56a-bc4c4d05df2f None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:53.034959 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1451743-e275-4cf5-a03e-da68fb4d0755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:53.035815 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1451743-e275-4cf5-a03e-da68fb4d0755 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:53.036090 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1451743-e275-4cf5-a03e-da68fb4d0755 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:53.036357 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1451743-e275-4cf5-a03e-da68fb4d0755 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:53.051400 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1451743-e275-4cf5-a03e-da68fb4d0755 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:30:53.057135 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-e1451743-e275-4cf5-a03e-da68fb4d0755 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 17:30:53.057135 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:30:53.057135 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:30:53.057135 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:30:53.057135 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:30:53.057135 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:30:53.057135 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:30:53.057135 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:30:53.057135 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:30:53.057135 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:30:53.057135 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:30:53.057135 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:30:53.057135 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:30:53.057135 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:30:53.057135 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:30:53.057135 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:30:53.058046 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:30:53.058046 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:30:53.058046 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:30:53.058046 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:30:53.058046 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:30:53.058046 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:30:53.058046 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:30:53.058046 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:30:53.058046 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:30:53.058046 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:30:53.058046 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:30:53.058046 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( Jul 28 17:30:53.058046 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:30:53.058046 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:30:53.058046 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:30:53.058046 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:30:53.058827 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:30:53.058827 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:30:53.058827 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:30:53.058827 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:30:53.058827 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:30:53.058827 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:30:53.058827 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:30:53.058827 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:30:53.058827 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:30:53.058827 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:30:53.058827 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 17:30:53.058827 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:30:53.058827 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 128/249] 10.222.0.89 () {56 vars in 1099 bytes} [Thu Jul 28 17:30:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:30:53.060480 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448866f9-08fd-45e7-b018-e65abdcdd0d2 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:53.060480 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448866f9-08fd-45e7-b018-e65abdcdd0d2 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:53.060480 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448866f9-08fd-45e7-b018-e65abdcdd0d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:53.085435 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-448866f9-08fd-45e7-b018-e65abdcdd0d2 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:30:53.086629 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-448866f9-08fd-45e7-b018-e65abdcdd0d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:30:53.111300 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:53.174999 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71cd658-0060-4fd9-a56a-bc4c4d05df2f None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:53.175846 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 129/250] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:30:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:30:53.176921 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448866f9-08fd-45e7-b018-e65abdcdd0d2 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:53.215947 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd4d8ec-4344-4fc7-8e3a-53dca1a79144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:53.216712 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd4d8ec-4344-4fc7-8e3a-53dca1a79144 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:53.217047 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd4d8ec-4344-4fc7-8e3a-53dca1a79144 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:53.217378 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd4d8ec-4344-4fc7-8e3a-53dca1a79144 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:53.219034 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd4d8ec-4344-4fc7-8e3a-53dca1a79144 None admin] RBAC: Adding query filter params (role.id=466df921c2db4456b37637c23382d92a, scope.project.id=16690c26262a497d9b62ac127ed8bf2e, user.id=b2c2dfb020434b11af87233d93f83e0f) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:30:53.219327 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd4d8ec-4344-4fc7-8e3a-53dca1a79144 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:53.221520 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd4d8ec-4344-4fc7-8e3a-53dca1a79144 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:53.228798 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 122/251] 10.222.0.89 () {56 vars in 1374 bytes} [Thu Jul 28 17:30:53 2022] GET /identity/v3/role_assignments?user.id=b2c2dfb020434b11af87233d93f83e0f&scope.project.id=16690c26262a497d9b62ac127ed8bf2e&role.id=466df921c2db4456b37637c23382d92a => generated 259 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:30:53.233580 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd4d8ec-4344-4fc7-8e3a-53dca1a79144 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:53.358041 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd4d8ec-4344-4fc7-8e3a-53dca1a79144 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:53.379500 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:53.402365 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378b3077-706e-4fab-aa8c-e5b8660d7585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:53.403765 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378b3077-706e-4fab-aa8c-e5b8660d7585 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:53.404313 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378b3077-706e-4fab-aa8c-e5b8660d7585 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:53.404783 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378b3077-706e-4fab-aa8c-e5b8660d7585 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:53.418497 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378b3077-706e-4fab-aa8c-e5b8660d7585 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:30:53.425195 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-378b3077-706e-4fab-aa8c-e5b8660d7585 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:30:53.425195 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:30:53.425195 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:30:53.425195 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:30:53.425195 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:30:53.425195 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:30:53.425195 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:30:53.425195 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:30:53.425195 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:30:53.425195 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:30:53.425195 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:30:53.425195 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:30:53.425195 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:30:53.425195 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:30:53.425195 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:30:53.425195 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:30:53.426134 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:30:53.426134 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:30:53.426134 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:30:53.426134 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:30:53.426134 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:30:53.426134 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:30:53.426134 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:30:53.426134 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:30:53.426134 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:30:53.426134 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:30:53.426134 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:30:53.426134 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:30:53.426134 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:30:53.426134 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:30:53.426134 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:30:53.426134 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:30:53.426908 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:30:53.426908 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:30:53.426908 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:30:53.426908 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:30:53.426908 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:30:53.426908 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:30:53.426908 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:30:53.426908 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:30:53.426908 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:30:53.426908 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:30:53.426908 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:30:53.426908 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:30:53.429676 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 123/252] 10.222.0.89 () {56 vars in 1096 bytes} [Thu Jul 28 17:30:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:30:53.442193 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378b3077-706e-4fab-aa8c-e5b8660d7585 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:53.447263 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448866f9-08fd-45e7-b018-e65abdcdd0d2 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:53.481561 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:53.558639 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01bb9ca-9fba-401f-8b50-7ea8adc381d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:53.559396 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01bb9ca-9fba-401f-8b50-7ea8adc381d1 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:53.559750 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01bb9ca-9fba-401f-8b50-7ea8adc381d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:53.560092 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01bb9ca-9fba-401f-8b50-7ea8adc381d1 None admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:53.561663 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01bb9ca-9fba-401f-8b50-7ea8adc381d1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:30:53.561891 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01bb9ca-9fba-401f-8b50-7ea8adc381d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:53.563503 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01bb9ca-9fba-401f-8b50-7ea8adc381d1 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:53.589263 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 130/253] 10.222.0.89 () {56 vars in 1101 bytes} [Thu Jul 28 17:30:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 412 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:30:53.594857 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01bb9ca-9fba-401f-8b50-7ea8adc381d1 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:53.600622 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378b3077-706e-4fab-aa8c-e5b8660d7585 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:53.647068 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3891db74-27a2-4b12-899a-c0cc57dccd7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:53.648027 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3891db74-27a2-4b12-899a-c0cc57dccd7a None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:53.648521 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3891db74-27a2-4b12-899a-c0cc57dccd7a None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:53.648990 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3891db74-27a2-4b12-899a-c0cc57dccd7a None admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:53.649926 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3891db74-27a2-4b12-899a-c0cc57dccd7a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:30:53.652550 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3891db74-27a2-4b12-899a-c0cc57dccd7a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:53.654236 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3891db74-27a2-4b12-899a-c0cc57dccd7a None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:53.674130 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 124/254] 10.222.0.89 () {56 vars in 1099 bytes} [Thu Jul 28 17:30:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:30:53.679799 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3891db74-27a2-4b12-899a-c0cc57dccd7a None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:53.771822 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01bb9ca-9fba-401f-8b50-7ea8adc381d1 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:53.801395 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:53.824852 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d01db3-7915-4a62-9b28-8826101c3e6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:53.826154 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d01db3-7915-4a62-9b28-8826101c3e6f None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:53.826704 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d01db3-7915-4a62-9b28-8826101c3e6f None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:53.828038 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d01db3-7915-4a62-9b28-8826101c3e6f None admin] PATH_INFO: `/v3/users/d763f6f23f304e44a2c1ac9028371ae8` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:53.831754 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d01db3-7915-4a62-9b28-8826101c3e6f None admin] RBAC: Authorizing `identity:get_user(user_id=d763f6f23f304e44a2c1ac9028371ae8)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:53.833463 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d01db3-7915-4a62-9b28-8826101c3e6f None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:53.835649 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 131/255] 10.222.0.89 () {56 vars in 1177 bytes} [Thu Jul 28 17:30:53 2022] GET /identity/v3/users/d763f6f23f304e44a2c1ac9028371ae8 => generated 297 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:30:53.841303 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d01db3-7915-4a62-9b28-8826101c3e6f None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:53.929592 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:53.940177 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3891db74-27a2-4b12-899a-c0cc57dccd7a None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:54.012709 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63effc70-ede0-4469-8f0d-6f686eb3122c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:54.013774 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63effc70-ede0-4469-8f0d-6f686eb3122c None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:54.014293 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63effc70-ede0-4469-8f0d-6f686eb3122c None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:54.014764 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63effc70-ede0-4469-8f0d-6f686eb3122c None admin] PATH_INFO: `/v3/users/2fb4d4a4ce11400dbfcb8b0f36a5b97c` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:54.016453 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63effc70-ede0-4469-8f0d-6f686eb3122c None admin] RBAC: Authorizing `identity:get_user(user_id=2fb4d4a4ce11400dbfcb8b0f36a5b97c)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:54.018157 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63effc70-ede0-4469-8f0d-6f686eb3122c None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:54.020004 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 125/256] 10.222.0.89 () {56 vars in 1177 bytes} [Thu Jul 28 17:30:53 2022] GET /identity/v3/users/2fb4d4a4ce11400dbfcb8b0f36a5b97c => generated 283 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:30:54.021350 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963b8a66-6193-499f-a1a4-59de3c5981fa None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:54.021831 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963b8a66-6193-499f-a1a4-59de3c5981fa None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:54.022413 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963b8a66-6193-499f-a1a4-59de3c5981fa None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:54.023888 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 126/257] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:30:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:30:54.029777 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963b8a66-6193-499f-a1a4-59de3c5981fa None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:54.031925 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d01db3-7915-4a62-9b28-8826101c3e6f None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:54.118475 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb39804-443f-4503-89d6-0feebef8d4e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:54.119436 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb39804-443f-4503-89d6-0feebef8d4e8 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:54.119787 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb39804-443f-4503-89d6-0feebef8d4e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:54.120154 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb39804-443f-4503-89d6-0feebef8d4e8 None admin] PATH_INFO: `/v3/projects/16690c26262a497d9b62ac127ed8bf2e` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:54.121677 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb39804-443f-4503-89d6-0feebef8d4e8 None admin] RBAC: Authorizing `identity:get_project(project_id=16690c26262a497d9b62ac127ed8bf2e)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:54.123195 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb39804-443f-4503-89d6-0feebef8d4e8 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:54.125741 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 132/258] 10.222.0.89 () {56 vars in 1186 bytes} [Thu Jul 28 17:30:53 2022] GET /identity/v3/projects/16690c26262a497d9b62ac127ed8bf2e => generated 302 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:30:54.126760 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e798996-e0ce-49d7-b18d-2b95ed09b047 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:54.127105 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e798996-e0ce-49d7-b18d-2b95ed09b047 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:54.127449 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e798996-e0ce-49d7-b18d-2b95ed09b047 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:54.178697 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-3e798996-e0ce-49d7-b18d-2b95ed09b047 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:30:54.179974 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-3e798996-e0ce-49d7-b18d-2b95ed09b047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:30:54.223522 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963b8a66-6193-499f-a1a4-59de3c5981fa None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:54.234462 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:54.266357 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459e7ba4-e65b-4726-88a4-e69c7093b15b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:54.266996 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459e7ba4-e65b-4726-88a4-e69c7093b15b None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:54.267279 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459e7ba4-e65b-4726-88a4-e69c7093b15b None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:54.267554 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459e7ba4-e65b-4726-88a4-e69c7093b15b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:54.347383 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459e7ba4-e65b-4726-88a4-e69c7093b15b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:54.349046 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459e7ba4-e65b-4726-88a4-e69c7093b15b None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:54.394872 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 133/259] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:30:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:30:54.395888 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e798996-e0ce-49d7-b18d-2b95ed09b047 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:54.467894 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 127/260] 10.222.0.89 () {58 vars in 1282 bytes} [Thu Jul 28 17:30:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 17:30:54.468997 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459e7ba4-e65b-4726-88a4-e69c7093b15b None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:54.486223 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:54.525548 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:54.642287 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459e7ba4-e65b-4726-88a4-e69c7093b15b None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:54.663350 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e798996-e0ce-49d7-b18d-2b95ed09b047 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:54.716196 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf050dbf-f61c-4d68-9f7e-b7f233d9b557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:54.716794 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf050dbf-f61c-4d68-9f7e-b7f233d9b557 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:54.717131 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf050dbf-f61c-4d68-9f7e-b7f233d9b557 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:54.717486 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf050dbf-f61c-4d68-9f7e-b7f233d9b557 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:54.728849 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf050dbf-f61c-4d68-9f7e-b7f233d9b557 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:30:54.730467 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54eb3574-9f6f-44cc-83cd-90a0ae683950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:54.731534 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eb3574-9f6f-44cc-83cd-90a0ae683950 None admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:54.732027 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eb3574-9f6f-44cc-83cd-90a0ae683950 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:54.732499 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eb3574-9f6f-44cc-83cd-90a0ae683950 None admin] PATH_INFO: `/v3/projects/16690c26262a497d9b62ac127ed8bf2e/users/d763f6f23f304e44a2c1ac9028371ae8/roles/11f8731756fe4caca13efa61b71b11c5` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:54.737182 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-bf050dbf-f61c-4d68-9f7e-b7f233d9b557 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 17:30:54.737182 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:30:54.737182 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:30:54.737182 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:30:54.737182 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:30:54.737182 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:30:54.737182 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:30:54.737182 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:30:54.737182 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:30:54.737182 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:30:54.737182 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:30:54.737182 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:30:54.737182 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:30:54.737182 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:30:54.737182 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:30:54.737182 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:30:54.738074 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:30:54.738074 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:30:54.738074 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:30:54.738074 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:30:54.738074 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:30:54.738074 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:30:54.738074 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:30:54.738074 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:30:54.738074 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:30:54.738074 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:30:54.738074 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:30:54.738074 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:30:54.738074 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:30:54.738074 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:30:54.738074 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:30:54.738074 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:30:54.738868 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:30:54.738868 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:30:54.738868 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:30:54.738868 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:30:54.738868 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:30:54.738868 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:30:54.738868 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:30:54.738868 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:30:54.738868 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:30:54.738868 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:30:54.738868 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 17:30:54.738868 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:30:54.738868 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 128/261] 10.222.0.89 () {56 vars in 1114 bytes} [Thu Jul 28 17:30:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:30:54.738868 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf050dbf-f61c-4d68-9f7e-b7f233d9b557 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:54.741866 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54eb3574-9f6f-44cc-83cd-90a0ae683950 None admin] RBAC: Authorizing `identity:create_grant(project_id=16690c26262a497d9b62ac127ed8bf2e, user_id=d763f6f23f304e44a2c1ac9028371ae8, role_id=11f8731756fe4caca13efa61b71b11c5)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:54.743350 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54eb3574-9f6f-44cc-83cd-90a0ae683950 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:54.768383 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 134/262] 10.222.0.89 () {58 vars in 1439 bytes} [Thu Jul 28 17:30:54 2022] PUT /identity/v3/projects/16690c26262a497d9b62ac127ed8bf2e/users/d763f6f23f304e44a2c1ac9028371ae8/roles/11f8731756fe4caca13efa61b71b11c5 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:30:54.769263 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54eb3574-9f6f-44cc-83cd-90a0ae683950 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:54.864775 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:54.967760 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf050dbf-f61c-4d68-9f7e-b7f233d9b557 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:54.984012 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54eb3574-9f6f-44cc-83cd-90a0ae683950 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:54.986440 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:55.027886 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb89bfb-7c79-4e8c-83f0-7e630f8ebd1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:55.029613 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb89bfb-7c79-4e8c-83f0-7e630f8ebd1a None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:55.030620 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb89bfb-7c79-4e8c-83f0-7e630f8ebd1a None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:55.031097 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb89bfb-7c79-4e8c-83f0-7e630f8ebd1a None admin] PATH_INFO: `/v3/projects/16690c26262a497d9b62ac127ed8bf2e` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:55.032684 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb89bfb-7c79-4e8c-83f0-7e630f8ebd1a None admin] RBAC: Authorizing `identity:get_project(project_id=16690c26262a497d9b62ac127ed8bf2e)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:55.034591 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb89bfb-7c79-4e8c-83f0-7e630f8ebd1a None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:55.036588 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 129/263] 10.222.0.89 () {56 vars in 1186 bytes} [Thu Jul 28 17:30:54 2022] GET /identity/v3/projects/16690c26262a497d9b62ac127ed8bf2e => generated 302 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:30:55.042920 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb89bfb-7c79-4e8c-83f0-7e630f8ebd1a None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:55.052821 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25357d87-2876-4d2e-965f-60772d239b96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:55.054935 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25357d87-2876-4d2e-965f-60772d239b96 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:55.056599 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25357d87-2876-4d2e-965f-60772d239b96 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:55.057078 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25357d87-2876-4d2e-965f-60772d239b96 None admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:55.058072 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25357d87-2876-4d2e-965f-60772d239b96 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:30:55.060979 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25357d87-2876-4d2e-965f-60772d239b96 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:55.063032 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25357d87-2876-4d2e-965f-60772d239b96 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:55.080009 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 135/264] 10.222.0.89 () {56 vars in 1111 bytes} [Thu Jul 28 17:30:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:30:55.084353 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25357d87-2876-4d2e-965f-60772d239b96 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:55.261304 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb89bfb-7c79-4e8c-83f0-7e630f8ebd1a None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:55.320191 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25357d87-2876-4d2e-965f-60772d239b96 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:55.327827 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052b62b7-f4da-4ecc-98ea-c9ed061238d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:55.328680 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052b62b7-f4da-4ecc-98ea-c9ed061238d2 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:55.329187 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052b62b7-f4da-4ecc-98ea-c9ed061238d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:55.329697 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052b62b7-f4da-4ecc-98ea-c9ed061238d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:55.330592 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052b62b7-f4da-4ecc-98ea-c9ed061238d2 None admin] RBAC: Adding query filter params (role.id=73e457231a824ea98c28d22610be640c, scope.project.id=16690c26262a497d9b62ac127ed8bf2e, user.id=2fb4d4a4ce11400dbfcb8b0f36a5b97c) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:30:55.330913 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052b62b7-f4da-4ecc-98ea-c9ed061238d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:55.332387 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052b62b7-f4da-4ecc-98ea-c9ed061238d2 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:55.343539 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:55.351826 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 130/265] 10.222.0.89 () {56 vars in 1374 bytes} [Thu Jul 28 17:30:55 2022] GET /identity/v3/role_assignments?user.id=2fb4d4a4ce11400dbfcb8b0f36a5b97c&scope.project.id=16690c26262a497d9b62ac127ed8bf2e&role.id=73e457231a824ea98c28d22610be640c => generated 608 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:30:55.393343 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fb09d6-3bca-4450-988c-4253ddb4ff80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:55.394001 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fb09d6-3bca-4450-988c-4253ddb4ff80 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:55.394385 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fb09d6-3bca-4450-988c-4253ddb4ff80 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:55.394701 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fb09d6-3bca-4450-988c-4253ddb4ff80 None admin] PATH_INFO: `/v3/users/2fb4d4a4ce11400dbfcb8b0f36a5b97c` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:55.402701 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fb09d6-3bca-4450-988c-4253ddb4ff80 None admin] RBAC: Authorizing `identity:get_user(user_id=2fb4d4a4ce11400dbfcb8b0f36a5b97c)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:55.404309 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fb09d6-3bca-4450-988c-4253ddb4ff80 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:55.407271 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 136/266] 10.222.0.89 () {56 vars in 1177 bytes} [Thu Jul 28 17:30:55 2022] GET /identity/v3/users/2fb4d4a4ce11400dbfcb8b0f36a5b97c => generated 283 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:30:55.412614 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052b62b7-f4da-4ecc-98ea-c9ed061238d2 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:55.553848 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7d5451-f3d7-4f0d-a283-44370c5d069f None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:55.554142 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7d5451-f3d7-4f0d-a283-44370c5d069f None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:55.554305 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7d5451-f3d7-4f0d-a283-44370c5d069f None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:55.555939 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 137/267] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:30:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:30:55.561948 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb721b8-9df9-45cd-a9e3-130484ab3d09 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:55.562320 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb721b8-9df9-45cd-a9e3-130484ab3d09 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:55.562599 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb721b8-9df9-45cd-a9e3-130484ab3d09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:55.583228 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-5cb721b8-9df9-45cd-a9e3-130484ab3d09 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:30:55.584890 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-5cb721b8-9df9-45cd-a9e3-130484ab3d09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:30:55.590897 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052b62b7-f4da-4ecc-98ea-c9ed061238d2 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:55.644357 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:55.654598 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d445cf7-f48a-4395-a4c3-3e491526c425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:55.656003 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d445cf7-f48a-4395-a4c3-3e491526c425 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:55.656285 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d445cf7-f48a-4395-a4c3-3e491526c425 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:55.656560 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d445cf7-f48a-4395-a4c3-3e491526c425 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:55.710143 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 138/268] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:30:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 17:30:55.744900 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb721b8-9df9-45cd-a9e3-130484ab3d09 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:55.763602 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:55.771234 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d445cf7-f48a-4395-a4c3-3e491526c425 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:55.773931 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d445cf7-f48a-4395-a4c3-3e491526c425 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:55.905975 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 131/269] 10.222.0.89 () {58 vars in 1282 bytes} [Thu Jul 28 17:30:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 494 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 17:30:55.908020 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ede87a-a737-4cd7-92be-7beac1deb9f7 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:55.908403 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ede87a-a737-4cd7-92be-7beac1deb9f7 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:55.908757 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ede87a-a737-4cd7-92be-7beac1deb9f7 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:55.910892 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 132/270] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:30:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:30:55.911719 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ede87a-a737-4cd7-92be-7beac1deb9f7 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:55.944017 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb721b8-9df9-45cd-a9e3-130484ab3d09 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:55.982220 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a7a868-c185-474b-af91-5882c1f997fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:55.982869 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a7a868-c185-474b-af91-5882c1f997fe None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:55.983369 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a7a868-c185-474b-af91-5882c1f997fe None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:55.983643 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a7a868-c185-474b-af91-5882c1f997fe None admin] PATH_INFO: `/v3/roles/member` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:55.996171 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a7a868-c185-474b-af91-5882c1f997fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:30:56.002349 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-c8a7a868-c185-474b-af91-5882c1f997fe None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 17:30:56.002349 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:30:56.002349 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:30:56.002349 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:30:56.002349 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:30:56.002349 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:30:56.002349 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:30:56.002349 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:30:56.002349 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:30:56.002349 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:30:56.002349 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:30:56.002349 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:30:56.002349 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:30:56.002349 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:30:56.002349 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:30:56.002349 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:30:56.003234 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:30:56.003234 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:30:56.003234 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:30:56.003234 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:30:56.003234 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:30:56.003234 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:30:56.003234 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:30:56.003234 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:30:56.003234 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:30:56.003234 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:30:56.003234 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:30:56.003234 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( Jul 28 17:30:56.003234 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:30:56.003234 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:30:56.003234 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:30:56.003234 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:30:56.004000 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:30:56.004000 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:30:56.004000 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:30:56.004000 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:30:56.004000 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:30:56.004000 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:30:56.004000 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:30:56.004000 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:30:56.004000 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:30:56.004000 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:30:56.004000 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 17:30:56.004000 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:30:56.004000 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 139/271] 10.222.0.89 () {56 vars in 1099 bytes} [Thu Jul 28 17:30:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:30:56.005107 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1323904e-6046-4bd4-af98-07e0ffafa93c None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:56.009566 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1323904e-6046-4bd4-af98-07e0ffafa93c None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:56.009862 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1323904e-6046-4bd4-af98-07e0ffafa93c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:56.033626 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-1323904e-6046-4bd4-af98-07e0ffafa93c None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:30:56.034444 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:56.035439 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-1323904e-6046-4bd4-af98-07e0ffafa93c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:30:56.038931 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ede87a-a737-4cd7-92be-7beac1deb9f7 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:56.059095 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:56.086597 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ea6d7b-13ec-46e1-8aad-a099e440c30b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:56.087212 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ea6d7b-13ec-46e1-8aad-a099e440c30b None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:56.087758 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ea6d7b-13ec-46e1-8aad-a099e440c30b None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:56.088034 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ea6d7b-13ec-46e1-8aad-a099e440c30b None admin] PATH_INFO: `/v3/projects/16690c26262a497d9b62ac127ed8bf2e` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:56.089343 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ea6d7b-13ec-46e1-8aad-a099e440c30b None admin] RBAC: Authorizing `identity:get_project(project_id=16690c26262a497d9b62ac127ed8bf2e)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:56.090801 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ea6d7b-13ec-46e1-8aad-a099e440c30b None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:56.093110 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 133/272] 10.222.0.89 () {56 vars in 1186 bytes} [Thu Jul 28 17:30:55 2022] GET /identity/v3/projects/16690c26262a497d9b62ac127ed8bf2e => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:30:56.094230 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ea6d7b-13ec-46e1-8aad-a099e440c30b None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:56.124888 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 140/273] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:30:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:30:56.134687 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6b6b88-3105-4139-98b5-f0fa7afc710c None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:56.134975 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6b6b88-3105-4139-98b5-f0fa7afc710c None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:56.135247 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6b6b88-3105-4139-98b5-f0fa7afc710c None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:56.137211 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 141/274] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:30:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:30:56.143723 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6b6b88-3105-4139-98b5-f0fa7afc710c None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:56.202884 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ea6d7b-13ec-46e1-8aad-a099e440c30b None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:56.240229 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fa80f1-476a-4bab-98bd-715f0cc5f319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:56.240854 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fa80f1-476a-4bab-98bd-715f0cc5f319 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:56.241139 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fa80f1-476a-4bab-98bd-715f0cc5f319 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:56.241447 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fa80f1-476a-4bab-98bd-715f0cc5f319 None admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:56.242164 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fa80f1-476a-4bab-98bd-715f0cc5f319 None admin] RBAC: Adding query filter params (name=member) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:30:56.242472 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fa80f1-476a-4bab-98bd-715f0cc5f319 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:56.243852 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fa80f1-476a-4bab-98bd-715f0cc5f319 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:56.253915 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 134/275] 10.222.0.89 () {56 vars in 1101 bytes} [Thu Jul 28 17:30:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:30:56.255129 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505882fb-3dce-41c3-8029-764f8864585e None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:56.255416 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505882fb-3dce-41c3-8029-764f8864585e None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:56.255688 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505882fb-3dce-41c3-8029-764f8864585e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:56.260623 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6b6b88-3105-4139-98b5-f0fa7afc710c None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:56.274200 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-505882fb-3dce-41c3-8029-764f8864585e None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:30:56.275421 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-505882fb-3dce-41c3-8029-764f8864585e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:30:56.296912 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cdab8e-d722-446f-9c6b-1c0bfa4ffb6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:56.297573 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cdab8e-d722-446f-9c6b-1c0bfa4ffb6a None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:56.297858 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cdab8e-d722-446f-9c6b-1c0bfa4ffb6a None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:56.298134 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cdab8e-d722-446f-9c6b-1c0bfa4ffb6a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:56.298935 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cdab8e-d722-446f-9c6b-1c0bfa4ffb6a None admin] RBAC: Adding query filter params (role.id=895fd016dffb41e7b0205c9a55a4e0f8, scope.project.id=16690c26262a497d9b62ac127ed8bf2e, user.id=2fb4d4a4ce11400dbfcb8b0f36a5b97c) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:30:56.299247 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cdab8e-d722-446f-9c6b-1c0bfa4ffb6a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:56.300697 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cdab8e-d722-446f-9c6b-1c0bfa4ffb6a None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:56.301608 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:56.307534 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 142/276] 10.222.0.89 () {56 vars in 1374 bytes} [Thu Jul 28 17:30:56 2022] GET /identity/v3/role_assignments?user.id=2fb4d4a4ce11400dbfcb8b0f36a5b97c&scope.project.id=16690c26262a497d9b62ac127ed8bf2e&role.id=895fd016dffb41e7b0205c9a55a4e0f8 => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:30:56.310018 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cdab8e-d722-446f-9c6b-1c0bfa4ffb6a None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:56.339445 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:56.359492 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 135/277] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:30:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:30:56.360217 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505882fb-3dce-41c3-8029-764f8864585e None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:56.453466 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cdab8e-d722-446f-9c6b-1c0bfa4ffb6a None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:56.492722 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdb01f6-c809-462f-b117-828dc1c5e2ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:56.493400 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdb01f6-c809-462f-b117-828dc1c5e2ab None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:56.493679 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdb01f6-c809-462f-b117-828dc1c5e2ab None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:56.494201 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdb01f6-c809-462f-b117-828dc1c5e2ab None admin] PATH_INFO: `/v3/domains/default` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:56.495695 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdb01f6-c809-462f-b117-828dc1c5e2ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:56.497142 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdb01f6-c809-462f-b117-828dc1c5e2ab None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:56.499708 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505882fb-3dce-41c3-8029-764f8864585e None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:56.500778 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 143/278] 10.222.0.89 () {56 vars in 1108 bytes} [Thu Jul 28 17:30:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:30:56.501000 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdb01f6-c809-462f-b117-828dc1c5e2ab None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:56.544043 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f0893a-4984-4334-995e-9bbca4343810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:56.544700 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f0893a-4984-4334-995e-9bbca4343810 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:56.544976 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f0893a-4984-4334-995e-9bbca4343810 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:56.545287 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f0893a-4984-4334-995e-9bbca4343810 None admin] PATH_INFO: `/v3/users/c4b04cd90f924f469a7b0b254e5026d8` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:56.566349 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f0893a-4984-4334-995e-9bbca4343810 None admin] RBAC: Authorizing `identity:get_user(user_id=c4b04cd90f924f469a7b0b254e5026d8)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:56.568659 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f0893a-4984-4334-995e-9bbca4343810 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:56.571072 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 136/279] 10.222.0.89 () {56 vars in 1177 bytes} [Thu Jul 28 17:30:56 2022] GET /identity/v3/users/c4b04cd90f924f469a7b0b254e5026d8 => generated 293 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:30:56.572000 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f0893a-4984-4334-995e-9bbca4343810 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:56.597833 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:56.610933 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:56.630436 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdb01f6-c809-462f-b117-828dc1c5e2ab None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:56.667965 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1cc91d-8e9b-4124-8432-e864edf884b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:56.668610 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1cc91d-8e9b-4124-8432-e864edf884b4 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:56.668893 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1cc91d-8e9b-4124-8432-e864edf884b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:56.669187 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1cc91d-8e9b-4124-8432-e864edf884b4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:56.681362 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1cc91d-8e9b-4124-8432-e864edf884b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:30:56.686892 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-2e1cc91d-8e9b-4124-8432-e864edf884b4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 17:30:56.686892 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:30:56.686892 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:30:56.686892 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:30:56.686892 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:30:56.686892 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:30:56.686892 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:30:56.686892 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:30:56.686892 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:30:56.686892 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:30:56.686892 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:30:56.686892 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:30:56.686892 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:30:56.686892 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:30:56.686892 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:30:56.686892 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:30:56.687858 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:30:56.687858 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:30:56.687858 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:30:56.687858 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:30:56.687858 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:30:56.687858 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:30:56.687858 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:30:56.687858 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:30:56.687858 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:30:56.687858 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:30:56.687858 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:30:56.687858 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( Jul 28 17:30:56.687858 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:30:56.687858 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:30:56.687858 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:30:56.687858 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:30:56.688636 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:30:56.688636 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:30:56.688636 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:30:56.688636 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:30:56.688636 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:30:56.688636 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:30:56.688636 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:30:56.688636 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:30:56.688636 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:30:56.688636 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:30:56.688636 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 17:30:56.688636 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:30:56.688636 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 144/280] 10.222.0.89 () {56 vars in 1099 bytes} [Thu Jul 28 17:30:56 2022] GET /identity/v3/roles/reader => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:30:56.689539 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1cc91d-8e9b-4124-8432-e864edf884b4 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:56.697735 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f0893a-4984-4334-995e-9bbca4343810 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:56.732968 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a460db-6d5f-460b-b45a-43d0d73c57ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:56.734212 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a460db-6d5f-460b-b45a-43d0d73c57ea None admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:56.734591 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a460db-6d5f-460b-b45a-43d0d73c57ea None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:56.734861 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a460db-6d5f-460b-b45a-43d0d73c57ea None admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:56.735536 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a460db-6d5f-460b-b45a-43d0d73c57ea None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:56.736959 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a460db-6d5f-460b-b45a-43d0d73c57ea None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:56.760092 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 137/281] 10.222.0.89 () {60 vars in 1132 bytes} [Thu Jul 28 17:30:56 2022] POST /identity/v3/users => generated 293 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:30:56.761025 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a460db-6d5f-460b-b45a-43d0d73c57ea None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:56.824683 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1cc91d-8e9b-4124-8432-e864edf884b4 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:56.857598 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:56.875338 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bd84d4-2d37-44fa-83bc-3656ee561b05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:56.876254 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bd84d4-2d37-44fa-83bc-3656ee561b05 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:56.876683 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bd84d4-2d37-44fa-83bc-3656ee561b05 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:56.877104 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bd84d4-2d37-44fa-83bc-3656ee561b05 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:56.878930 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bd84d4-2d37-44fa-83bc-3656ee561b05 None admin] RBAC: Adding query filter params (role.id=11f8731756fe4caca13efa61b71b11c5, user.id=c4b04cd90f924f469a7b0b254e5026d8, scope.system=all) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:30:56.879731 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bd84d4-2d37-44fa-83bc-3656ee561b05 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:56.882077 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bd84d4-2d37-44fa-83bc-3656ee561b05 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:56.890033 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 145/282] 10.222.0.89 () {56 vars in 1308 bytes} [Thu Jul 28 17:30:56 2022] GET /identity/v3/role_assignments?user.id=c4b04cd90f924f469a7b0b254e5026d8&scope.system=all&role.id=11f8731756fe4caca13efa61b71b11c5 => generated 226 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:30:56.895053 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a460db-6d5f-460b-b45a-43d0d73c57ea None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:56.939713 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b5e919-8c24-4cfd-8460-0e772dd31b3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:56.940687 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b5e919-8c24-4cfd-8460-0e772dd31b3b None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:56.941184 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b5e919-8c24-4cfd-8460-0e772dd31b3b None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:56.941661 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b5e919-8c24-4cfd-8460-0e772dd31b3b None admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:56.942448 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b5e919-8c24-4cfd-8460-0e772dd31b3b None admin] RBAC: Adding query filter params (name=reader) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:30:56.942762 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b5e919-8c24-4cfd-8460-0e772dd31b3b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:56.944191 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b5e919-8c24-4cfd-8460-0e772dd31b3b None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:56.954540 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 138/283] 10.222.0.89 () {56 vars in 1101 bytes} [Thu Jul 28 17:30:56 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:30:56.960274 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bd84d4-2d37-44fa-83bc-3656ee561b05 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:56.973222 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:57.079012 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bd84d4-2d37-44fa-83bc-3656ee561b05 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:57.118187 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591c5279-bce7-4160-a302-ea31b21f1e24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:57.118763 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591c5279-bce7-4160-a302-ea31b21f1e24 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:57.119303 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591c5279-bce7-4160-a302-ea31b21f1e24 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:57.119581 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591c5279-bce7-4160-a302-ea31b21f1e24 None admin] PATH_INFO: `/v3/users/b2c2dfb020434b11af87233d93f83e0f` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:57.121796 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591c5279-bce7-4160-a302-ea31b21f1e24 None admin] RBAC: Authorizing `identity:get_user(user_id=b2c2dfb020434b11af87233d93f83e0f)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:57.123264 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591c5279-bce7-4160-a302-ea31b21f1e24 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:57.125873 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 146/284] 10.222.0.89 () {56 vars in 1177 bytes} [Thu Jul 28 17:30:56 2022] GET /identity/v3/users/b2c2dfb020434b11af87233d93f83e0f => generated 297 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:30:57.131666 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b5e919-8c24-4cfd-8460-0e772dd31b3b None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:57.244933 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b5e919-8c24-4cfd-8460-0e772dd31b3b None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:57.283114 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f96d35-b518-43b5-93e4-7c86277ff01c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:57.283779 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f96d35-b518-43b5-93e4-7c86277ff01c None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:57.284066 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f96d35-b518-43b5-93e4-7c86277ff01c None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:57.284343 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f96d35-b518-43b5-93e4-7c86277ff01c None admin] PATH_INFO: `/v3/projects/16690c26262a497d9b62ac127ed8bf2e` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:57.285570 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f96d35-b518-43b5-93e4-7c86277ff01c None admin] RBAC: Authorizing `identity:get_project(project_id=16690c26262a497d9b62ac127ed8bf2e)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:57.287022 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f96d35-b518-43b5-93e4-7c86277ff01c None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:57.289104 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 139/285] 10.222.0.89 () {56 vars in 1186 bytes} [Thu Jul 28 17:30:57 2022] GET /identity/v3/projects/16690c26262a497d9b62ac127ed8bf2e => generated 302 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:30:57.294761 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591c5279-bce7-4160-a302-ea31b21f1e24 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:57.354708 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76f92b8-e65c-4c91-90ba-44a1d6df8c58 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:57.355010 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76f92b8-e65c-4c91-90ba-44a1d6df8c58 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:57.355262 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76f92b8-e65c-4c91-90ba-44a1d6df8c58 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:57.356750 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 140/286] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:30:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 17:30:57.362114 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a720cf14-dbee-42c4-9afa-3905f7ae023c None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:57.362403 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a720cf14-dbee-42c4-9afa-3905f7ae023c None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:57.362672 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a720cf14-dbee-42c4-9afa-3905f7ae023c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:57.370511 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:57.384554 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-a720cf14-dbee-42c4-9afa-3905f7ae023c None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:30:57.386241 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-a720cf14-dbee-42c4-9afa-3905f7ae023c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:30:57.406534 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:57.436199 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591c5279-bce7-4160-a302-ea31b21f1e24 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:57.481493 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e67af10-983f-499a-9bd3-c7ceb6d605db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:57.482537 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e67af10-983f-499a-9bd3-c7ceb6d605db None admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:57.483791 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e67af10-983f-499a-9bd3-c7ceb6d605db None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:57.484302 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e67af10-983f-499a-9bd3-c7ceb6d605db None admin] PATH_INFO: `/v3/projects/16690c26262a497d9b62ac127ed8bf2e/users/b2c2dfb020434b11af87233d93f83e0f/roles/466df921c2db4456b37637c23382d92a` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:57.488028 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e67af10-983f-499a-9bd3-c7ceb6d605db None admin] RBAC: Authorizing `identity:create_grant(project_id=16690c26262a497d9b62ac127ed8bf2e, user_id=b2c2dfb020434b11af87233d93f83e0f, role_id=466df921c2db4456b37637c23382d92a)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:57.491013 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e67af10-983f-499a-9bd3-c7ceb6d605db None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:57.507137 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 141/287] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:30:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 17:30:57.517278 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 147/288] 10.222.0.89 () {58 vars in 1439 bytes} [Thu Jul 28 17:30:57 2022] PUT /identity/v3/projects/16690c26262a497d9b62ac127ed8bf2e/users/b2c2dfb020434b11af87233d93f83e0f/roles/466df921c2db4456b37637c23382d92a => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:30:57.533360 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a720cf14-dbee-42c4-9afa-3905f7ae023c None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:57.679335 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a720cf14-dbee-42c4-9afa-3905f7ae023c None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:57.723945 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60de744-e4ed-472b-861f-80e78b060ffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:57.725478 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60de744-e4ed-472b-861f-80e78b060ffd None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:57.725985 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60de744-e4ed-472b-861f-80e78b060ffd None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:57.726455 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60de744-e4ed-472b-861f-80e78b060ffd None admin] PATH_INFO: `/v3/roles/member` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:57.737526 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:57.750654 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60de744-e4ed-472b-861f-80e78b060ffd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:30:57.755923 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-c60de744-e4ed-472b-861f-80e78b060ffd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 17:30:57.755923 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:30:57.755923 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:30:57.755923 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:30:57.755923 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:30:57.755923 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:30:57.755923 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:30:57.755923 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:30:57.755923 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:30:57.755923 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:30:57.755923 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:30:57.755923 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:30:57.755923 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:30:57.755923 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:30:57.755923 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:30:57.755923 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:30:57.756810 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:30:57.756810 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:30:57.756810 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:30:57.756810 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:30:57.756810 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:30:57.756810 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:30:57.756810 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:30:57.756810 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:30:57.756810 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:30:57.756810 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:30:57.756810 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:30:57.756810 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:30:57.756810 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:30:57.756810 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:30:57.756810 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:30:57.756810 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:30:57.757658 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:30:57.757658 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:30:57.757658 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:30:57.757658 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:30:57.757658 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:30:57.757658 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:30:57.757658 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:30:57.757658 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:30:57.757658 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:30:57.757658 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:30:57.757658 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 17:30:57.757658 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:30:57.757658 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 142/289] 10.222.0.89 () {56 vars in 1099 bytes} [Thu Jul 28 17:30:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:30:57.765310 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e67af10-983f-499a-9bd3-c7ceb6d605db None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:57.879677 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e67af10-983f-499a-9bd3-c7ceb6d605db None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:57.915989 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5d957a-34bf-44db-b615-909c82d1c3ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:57.916610 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5d957a-34bf-44db-b615-909c82d1c3ec None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:57.916890 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5d957a-34bf-44db-b615-909c82d1c3ec None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:57.917344 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5d957a-34bf-44db-b615-909c82d1c3ec None admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:57.918126 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5d957a-34bf-44db-b615-909c82d1c3ec None admin] RBAC: Adding query filter params (name=member) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:30:57.918429 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5d957a-34bf-44db-b615-909c82d1c3ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:57.919804 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5d957a-34bf-44db-b615-909c82d1c3ec None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:57.923565 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:57.931567 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 148/290] 10.222.0.89 () {56 vars in 1101 bytes} [Thu Jul 28 17:30:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:30:57.937672 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60de744-e4ed-472b-861f-80e78b060ffd None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:58.070875 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60de744-e4ed-472b-861f-80e78b060ffd None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:58.106026 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944c2160-be5f-4746-acd1-22b41569aee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:58.106691 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944c2160-be5f-4746-acd1-22b41569aee2 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:58.106971 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944c2160-be5f-4746-acd1-22b41569aee2 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:58.107243 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944c2160-be5f-4746-acd1-22b41569aee2 None admin] PATH_INFO: `/v3/users/d763f6f23f304e44a2c1ac9028371ae8` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:58.109097 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944c2160-be5f-4746-acd1-22b41569aee2 None admin] RBAC: Authorizing `identity:get_user(user_id=d763f6f23f304e44a2c1ac9028371ae8)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:58.110563 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944c2160-be5f-4746-acd1-22b41569aee2 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:58.112543 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 143/291] 10.222.0.89 () {56 vars in 1177 bytes} [Thu Jul 28 17:30:57 2022] GET /identity/v3/users/d763f6f23f304e44a2c1ac9028371ae8 => generated 297 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:30:58.118209 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5d957a-34bf-44db-b615-909c82d1c3ec None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:58.253494 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5d957a-34bf-44db-b615-909c82d1c3ec None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:58.293276 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2373554-abda-4f9d-99d4-1d25137f63a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:58.293889 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2373554-abda-4f9d-99d4-1d25137f63a5 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:58.294164 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2373554-abda-4f9d-99d4-1d25137f63a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:58.294434 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2373554-abda-4f9d-99d4-1d25137f63a5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:58.369649 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:58.376109 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2373554-abda-4f9d-99d4-1d25137f63a5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:58.377617 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2373554-abda-4f9d-99d4-1d25137f63a5 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:58.487667 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 149/292] 10.222.0.89 () {58 vars in 1282 bytes} [Thu Jul 28 17:30:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 17:30:58.494629 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944c2160-be5f-4746-acd1-22b41569aee2 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:58.573615 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:58.620589 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944c2160-be5f-4746-acd1-22b41569aee2 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:58.662581 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4339fc6-e288-4904-a30b-f80d454c301f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:58.663239 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4339fc6-e288-4904-a30b-f80d454c301f None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:58.663517 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4339fc6-e288-4904-a30b-f80d454c301f None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:58.663786 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4339fc6-e288-4904-a30b-f80d454c301f None admin] PATH_INFO: `/v3/projects/16690c26262a497d9b62ac127ed8bf2e` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:58.664960 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4339fc6-e288-4904-a30b-f80d454c301f None admin] RBAC: Authorizing `identity:get_project(project_id=16690c26262a497d9b62ac127ed8bf2e)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:58.666435 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4339fc6-e288-4904-a30b-f80d454c301f None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:58.668293 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 144/293] 10.222.0.89 () {56 vars in 1186 bytes} [Thu Jul 28 17:30:58 2022] GET /identity/v3/projects/16690c26262a497d9b62ac127ed8bf2e => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:30:58.675084 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2373554-abda-4f9d-99d4-1d25137f63a5 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:58.816531 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:58.821381 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2373554-abda-4f9d-99d4-1d25137f63a5 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:58.861725 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e81411-6054-4daf-906d-7422f34a62e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:58.862351 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e81411-6054-4daf-906d-7422f34a62e6 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:58.862629 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e81411-6054-4daf-906d-7422f34a62e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:58.862900 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e81411-6054-4daf-906d-7422f34a62e6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:58.863695 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e81411-6054-4daf-906d-7422f34a62e6 None admin] RBAC: Adding query filter params (role.id=11f8731756fe4caca13efa61b71b11c5, scope.project.id=16690c26262a497d9b62ac127ed8bf2e, user.id=d763f6f23f304e44a2c1ac9028371ae8) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:30:58.863997 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e81411-6054-4daf-906d-7422f34a62e6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:58.865480 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e81411-6054-4daf-906d-7422f34a62e6 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:58.872743 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 150/294] 10.222.0.89 () {56 vars in 1374 bytes} [Thu Jul 28 17:30:58 2022] GET /identity/v3/role_assignments?user.id=d763f6f23f304e44a2c1ac9028371ae8&scope.project.id=16690c26262a497d9b62ac127ed8bf2e&role.id=11f8731756fe4caca13efa61b71b11c5 => generated 608 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:30:59.266092 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ff6ea9-0cab-4980-86ed-91a0616908e5 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:59.266608 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ff6ea9-0cab-4980-86ed-91a0616908e5 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:59.266608 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ff6ea9-0cab-4980-86ed-91a0616908e5 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:59.268278 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 145/295] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:30:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:30:59.273357 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff779ec4-3056-48cf-9bc8-7b030e0033a1 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:59.273862 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff779ec4-3056-48cf-9bc8-7b030e0033a1 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:59.274124 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff779ec4-3056-48cf-9bc8-7b030e0033a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:59.291864 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aacb8c-694e-4bb6-ba90-f6a348c12866 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:59.292157 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aacb8c-694e-4bb6-ba90-f6a348c12866 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:59.292406 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aacb8c-694e-4bb6-ba90-f6a348c12866 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:59.296579 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 146/296] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:30:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 17:30:59.297815 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-ff779ec4-3056-48cf-9bc8-7b030e0033a1 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:30:59.299266 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-ff779ec4-3056-48cf-9bc8-7b030e0033a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:30:59.306428 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7881ab2-832a-493e-80e8-dd00e7a47c1b None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:59.306708 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7881ab2-832a-493e-80e8-dd00e7a47c1b None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:59.306982 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7881ab2-832a-493e-80e8-dd00e7a47c1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:59.338918 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-c7881ab2-832a-493e-80e8-dd00e7a47c1b None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:30:59.338918 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-c7881ab2-832a-493e-80e8-dd00e7a47c1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:30:59.403959 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 151/297] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:30:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:30:59.425904 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff779ec4-3056-48cf-9bc8-7b030e0033a1 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:59.430927 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 147/298] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:30:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 17:30:59.455443 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7881ab2-832a-493e-80e8-dd00e7a47c1b None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:59.523928 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:59.544401 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:59.556118 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff779ec4-3056-48cf-9bc8-7b030e0033a1 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:59.574142 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7881ab2-832a-493e-80e8-dd00e7a47c1b None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:59.605598 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d60779a-5938-4492-b597-9fb5ff5bd82a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:59.606550 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d60779a-5938-4492-b597-9fb5ff5bd82a None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:59.607038 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d60779a-5938-4492-b597-9fb5ff5bd82a None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:59.607564 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d60779a-5938-4492-b597-9fb5ff5bd82a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:59.617035 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0105b587-c145-45d5-bee8-7e6c67c5dc27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:59.617720 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0105b587-c145-45d5-bee8-7e6c67c5dc27 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:59.618001 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0105b587-c145-45d5-bee8-7e6c67c5dc27 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:59.618281 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0105b587-c145-45d5-bee8-7e6c67c5dc27 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:59.620654 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d60779a-5938-4492-b597-9fb5ff5bd82a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:30:59.626866 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-6d60779a-5938-4492-b597-9fb5ff5bd82a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 17:30:59.626866 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:30:59.626866 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:30:59.626866 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:30:59.626866 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:30:59.626866 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:30:59.626866 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:30:59.626866 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:30:59.626866 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:30:59.626866 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:30:59.626866 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:30:59.626866 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:30:59.626866 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:30:59.626866 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:30:59.626866 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:30:59.626866 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:30:59.627790 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:30:59.627790 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:30:59.627790 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:30:59.627790 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:30:59.627790 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:30:59.627790 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:30:59.627790 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:30:59.627790 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:30:59.627790 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:30:59.627790 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:30:59.627790 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:30:59.627790 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( Jul 28 17:30:59.627790 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:30:59.627790 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:30:59.627790 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:30:59.627790 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:30:59.628576 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:30:59.628576 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:30:59.628576 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:30:59.628576 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:30:59.628576 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:30:59.628576 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:30:59.628576 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:30:59.628576 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:30:59.628576 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:30:59.628576 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:30:59.628576 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 17:30:59.628576 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:30:59.629855 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0105b587-c145-45d5-bee8-7e6c67c5dc27 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:30:59.631558 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 152/299] 10.222.0.89 () {56 vars in 1099 bytes} [Thu Jul 28 17:30:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:30:59.641221 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-0105b587-c145-45d5-bee8-7e6c67c5dc27 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 17:30:59.641221 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:30:59.641221 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:30:59.641221 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:30:59.641221 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:30:59.641221 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:30:59.641221 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:30:59.641221 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:30:59.641221 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:30:59.641221 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:30:59.641221 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:30:59.641221 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:30:59.641221 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:30:59.641221 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:30:59.641221 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:30:59.641221 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:30:59.642179 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:30:59.642179 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:30:59.642179 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:30:59.642179 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:30:59.642179 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:30:59.642179 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:30:59.642179 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:30:59.642179 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:30:59.642179 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:30:59.642179 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:30:59.642179 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:30:59.642179 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:30:59.642179 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:30:59.642179 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:30:59.642179 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:30:59.642179 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:30:59.643251 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:30:59.643251 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:30:59.643251 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:30:59.643251 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:30:59.643251 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:30:59.643251 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:30:59.643251 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:30:59.643251 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:30:59.643251 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:30:59.643251 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:30:59.643251 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 17:30:59.643251 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:30:59.643849 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d60779a-5938-4492-b597-9fb5ff5bd82a None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:59.643919 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 148/300] 10.222.0.89 () {56 vars in 1099 bytes} [Thu Jul 28 17:30:59 2022] GET /identity/v3/roles/reader => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:30:59.650182 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0105b587-c145-45d5-bee8-7e6c67c5dc27 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:59.761631 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0105b587-c145-45d5-bee8-7e6c67c5dc27 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:59.768217 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d60779a-5938-4492-b597-9fb5ff5bd82a None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:59.785908 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:59.803229 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a6cc76-3a73-4e37-a9be-45ac6352a503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:59.804357 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a6cc76-3a73-4e37-a9be-45ac6352a503 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:59.805006 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a6cc76-3a73-4e37-a9be-45ac6352a503 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:59.805510 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a6cc76-3a73-4e37-a9be-45ac6352a503 None admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:59.806444 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a6cc76-3a73-4e37-a9be-45ac6352a503 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:30:59.807057 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a6cc76-3a73-4e37-a9be-45ac6352a503 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:59.809044 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a6cc76-3a73-4e37-a9be-45ac6352a503 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:59.815192 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ae03c9-12b2-4322-a097-c2331f154927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:59.815894 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ae03c9-12b2-4322-a097-c2331f154927 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:59.816244 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ae03c9-12b2-4322-a097-c2331f154927 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:59.816579 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ae03c9-12b2-4322-a097-c2331f154927 None admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:30:59.817895 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ae03c9-12b2-4322-a097-c2331f154927 None admin] RBAC: Adding query filter params (name=member) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:30:59.818272 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ae03c9-12b2-4322-a097-c2331f154927 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:30:59.820025 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ae03c9-12b2-4322-a097-c2331f154927 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:30:59.822255 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 149/301] 10.222.0.89 () {56 vars in 1101 bytes} [Thu Jul 28 17:30:59 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:30:59.833082 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a6cc76-3a73-4e37-a9be-45ac6352a503 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:59.836519 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 153/302] 10.222.0.89 () {56 vars in 1101 bytes} [Thu Jul 28 17:30:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:30:59.841987 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ae03c9-12b2-4322-a097-c2331f154927 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:30:59.861732 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:30:59.955803 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a6cc76-3a73-4e37-a9be-45ac6352a503 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:59.972402 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ae03c9-12b2-4322-a097-c2331f154927 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:30:59.994857 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669b1ded-0fe4-48ef-8e5a-104a6872ca13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:30:59.995505 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669b1ded-0fe4-48ef-8e5a-104a6872ca13 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:30:59.995784 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669b1ded-0fe4-48ef-8e5a-104a6872ca13 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:30:59.996057 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669b1ded-0fe4-48ef-8e5a-104a6872ca13 None admin] PATH_INFO: `/v3/users/8f014dda3c444b289db91b94c6131013` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:00.008216 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb18011c-1e66-4bab-bb1a-9bcc7c3d4098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:00.008848 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb18011c-1e66-4bab-bb1a-9bcc7c3d4098 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:00.009122 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb18011c-1e66-4bab-bb1a-9bcc7c3d4098 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:00.009427 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb18011c-1e66-4bab-bb1a-9bcc7c3d4098 None admin] PATH_INFO: `/v3/users/c4b04cd90f924f469a7b0b254e5026d8` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:00.010581 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb18011c-1e66-4bab-bb1a-9bcc7c3d4098 None admin] RBAC: Authorizing `identity:get_user(user_id=c4b04cd90f924f469a7b0b254e5026d8)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:00.011814 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669b1ded-0fe4-48ef-8e5a-104a6872ca13 None admin] RBAC: Authorizing `identity:get_user(user_id=8f014dda3c444b289db91b94c6131013)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:00.012002 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb18011c-1e66-4bab-bb1a-9bcc7c3d4098 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:00.013283 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669b1ded-0fe4-48ef-8e5a-104a6872ca13 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:00.013992 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 154/303] 10.222.0.89 () {56 vars in 1177 bytes} [Thu Jul 28 17:30:59 2022] GET /identity/v3/users/c4b04cd90f924f469a7b0b254e5026d8 => generated 293 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:31:00.015464 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 150/304] 10.222.0.89 () {56 vars in 1177 bytes} [Thu Jul 28 17:30:59 2022] GET /identity/v3/users/8f014dda3c444b289db91b94c6131013 => generated 293 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:31:00.019266 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669b1ded-0fe4-48ef-8e5a-104a6872ca13 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:00.019485 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630b1fed-b1d8-4380-9947-b792dc02e11f None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:00.019748 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630b1fed-b1d8-4380-9947-b792dc02e11f None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:00.020018 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630b1fed-b1d8-4380-9947-b792dc02e11f None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:00.021325 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 155/305] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:31:00 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:31:00.022014 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630b1fed-b1d8-4380-9947-b792dc02e11f None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:00.068521 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:00.105787 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:00.128257 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630b1fed-b1d8-4380-9947-b792dc02e11f None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:00.134791 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669b1ded-0fe4-48ef-8e5a-104a6872ca13 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:00.163319 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681d819b-6699-46a2-ae79-abb5d09e812e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:00.163934 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681d819b-6699-46a2-ae79-abb5d09e812e None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:00.164208 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681d819b-6699-46a2-ae79-abb5d09e812e None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:00.164477 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681d819b-6699-46a2-ae79-abb5d09e812e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:00.165285 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681d819b-6699-46a2-ae79-abb5d09e812e None admin] RBAC: Adding query filter params (role.id=466df921c2db4456b37637c23382d92a, user.id=8f014dda3c444b289db91b94c6131013, scope.system=all) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:00.165589 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681d819b-6699-46a2-ae79-abb5d09e812e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:00.167067 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681d819b-6699-46a2-ae79-abb5d09e812e None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:00.170137 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f99ca6-808d-42f3-8f75-5624c56729b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:00.171496 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f99ca6-808d-42f3-8f75-5624c56729b0 None admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:00.171695 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f99ca6-808d-42f3-8f75-5624c56729b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:00.171970 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f99ca6-808d-42f3-8f75-5624c56729b0 None admin] PATH_INFO: `/v3/system/users/c4b04cd90f924f469a7b0b254e5026d8/roles/11f8731756fe4caca13efa61b71b11c5` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:00.173127 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 156/306] 10.222.0.89 () {56 vars in 1308 bytes} [Thu Jul 28 17:31:00 2022] GET /identity/v3/role_assignments?user.id=8f014dda3c444b289db91b94c6131013&scope.system=all&role.id=466df921c2db4456b37637c23382d92a => generated 226 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:00.173481 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f99ca6-808d-42f3-8f75-5624c56729b0 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=c4b04cd90f924f469a7b0b254e5026d8, role_id=11f8731756fe4caca13efa61b71b11c5)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:00.174270 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9fbd1a-1556-494a-8c00-745999c8494b None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:00.174550 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9fbd1a-1556-494a-8c00-745999c8494b None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:00.174820 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9fbd1a-1556-494a-8c00-745999c8494b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:00.174920 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f99ca6-808d-42f3-8f75-5624c56729b0 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:00.200710 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 151/307] 10.222.0.89 () {58 vars in 1334 bytes} [Thu Jul 28 17:31:00 2022] PUT /identity/v3/system/users/c4b04cd90f924f469a7b0b254e5026d8/roles/11f8731756fe4caca13efa61b71b11c5 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:31:00.200710 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb51028-f20c-47dd-b8d3-b25e79c2bbbd None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:00.200710 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb51028-f20c-47dd-b8d3-b25e79c2bbbd None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:00.200710 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb51028-f20c-47dd-b8d3-b25e79c2bbbd None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:00.200710 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 152/308] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:31:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:31:00.203050 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-ba9fbd1a-1556-494a-8c00-745999c8494b None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:31:00.204765 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-ba9fbd1a-1556-494a-8c00-745999c8494b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:31:00.205470 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd28b7ba-7479-4589-9e1c-1bd23bb99b60 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:00.205868 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd28b7ba-7479-4589-9e1c-1bd23bb99b60 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:00.206263 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd28b7ba-7479-4589-9e1c-1bd23bb99b60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:00.237289 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-fd28b7ba-7479-4589-9e1c-1bd23bb99b60 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:31:00.239626 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-fd28b7ba-7479-4589-9e1c-1bd23bb99b60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:31:00.330267 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 157/309] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:31:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:31:00.351359 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 153/310] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:31:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:31:00.353728 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9fbd1a-1556-494a-8c00-745999c8494b None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:00.375609 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd28b7ba-7479-4589-9e1c-1bd23bb99b60 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:00.414053 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:00.450628 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:00.476257 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9fbd1a-1556-494a-8c00-745999c8494b None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:00.500088 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd28b7ba-7479-4589-9e1c-1bd23bb99b60 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:00.522877 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53856ae5-d608-490d-847b-90b8b56e86e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:00.523583 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53856ae5-d608-490d-847b-90b8b56e86e2 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:00.523874 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53856ae5-d608-490d-847b-90b8b56e86e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:00.524296 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53856ae5-d608-490d-847b-90b8b56e86e2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:00.526006 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53856ae5-d608-490d-847b-90b8b56e86e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:00.527749 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53856ae5-d608-490d-847b-90b8b56e86e2 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:00.530976 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 158/311] 10.222.0.89 () {56 vars in 1108 bytes} [Thu Jul 28 17:31:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:00.537288 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53856ae5-d608-490d-847b-90b8b56e86e2 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:00.540873 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55d9f23-3bcb-48b7-98f0-a1a107a42cae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:00.541575 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55d9f23-3bcb-48b7-98f0-a1a107a42cae None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:00.541863 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55d9f23-3bcb-48b7-98f0-a1a107a42cae None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:00.542199 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55d9f23-3bcb-48b7-98f0-a1a107a42cae None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:00.553821 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55d9f23-3bcb-48b7-98f0-a1a107a42cae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:31:00.559133 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-e55d9f23-3bcb-48b7-98f0-a1a107a42cae None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 17:31:00.559133 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:00.559133 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:00.559133 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:00.559133 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:00.559133 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:00.559133 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:00.559133 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:00.559133 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:00.559133 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:00.559133 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:00.559133 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:00.559133 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:31:00.559133 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:31:00.559133 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:31:00.559133 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:31:00.560026 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:31:00.560026 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:31:00.560026 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:31:00.560026 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:31:00.560026 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:00.560026 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:00.560026 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:00.560026 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:00.560026 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:00.560026 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:00.560026 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:00.560026 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:00.560026 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:00.560026 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:00.560026 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:00.560026 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:00.560846 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:00.560846 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:00.560846 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:00.560846 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:00.560846 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:31:00.560846 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:31:00.560846 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:31:00.560846 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:31:00.560846 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:31:00.560846 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:31:00.560846 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 17:31:00.560846 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:31:00.560846 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 154/312] 10.222.0.89 () {56 vars in 1099 bytes} [Thu Jul 28 17:31:00 2022] GET /identity/v3/roles/reader => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:31:00.567210 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55d9f23-3bcb-48b7-98f0-a1a107a42cae None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:00.658252 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53856ae5-d608-490d-847b-90b8b56e86e2 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:00.699248 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fdd65a-d965-489c-9666-4a458a21d2f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:00.700229 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:00.700783 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fdd65a-d965-489c-9666-4a458a21d2f5 None admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:00.702182 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fdd65a-d965-489c-9666-4a458a21d2f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:00.702886 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fdd65a-d965-489c-9666-4a458a21d2f5 None admin] PATH_INFO: `/v3/groups` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:00.704239 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fdd65a-d965-489c-9666-4a458a21d2f5 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:00.706394 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fdd65a-d965-489c-9666-4a458a21d2f5 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:00.707776 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55d9f23-3bcb-48b7-98f0-a1a107a42cae None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:00.724789 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 159/313] 10.222.0.89 () {60 vars in 1134 bytes} [Thu Jul 28 17:31:00 2022] POST /identity/v3/groups => generated 229 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 17:31:00.747260 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f2f141-0a07-49ce-a1ee-5db07a161f38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:00.747909 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f2f141-0a07-49ce-a1ee-5db07a161f38 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:00.748188 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f2f141-0a07-49ce-a1ee-5db07a161f38 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:00.748465 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f2f141-0a07-49ce-a1ee-5db07a161f38 None admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:00.749910 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f2f141-0a07-49ce-a1ee-5db07a161f38 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:00.750219 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f2f141-0a07-49ce-a1ee-5db07a161f38 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:00.751681 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f2f141-0a07-49ce-a1ee-5db07a161f38 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:00.761465 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 155/314] 10.222.0.89 () {56 vars in 1101 bytes} [Thu Jul 28 17:31:00 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:31:00.767893 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fdd65a-d965-489c-9666-4a458a21d2f5 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:00.772717 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:00.890179 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fdd65a-d965-489c-9666-4a458a21d2f5 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:00.931682 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa8bc84-93ec-422e-aed4-fa4b9ff793cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:00.932659 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa8bc84-93ec-422e-aed4-fa4b9ff793cb None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:00.933149 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa8bc84-93ec-422e-aed4-fa4b9ff793cb None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:00.933676 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa8bc84-93ec-422e-aed4-fa4b9ff793cb None admin] PATH_INFO: `/v3/users/b2c2dfb020434b11af87233d93f83e0f` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:00.935098 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa8bc84-93ec-422e-aed4-fa4b9ff793cb None admin] RBAC: Authorizing `identity:get_user(user_id=b2c2dfb020434b11af87233d93f83e0f)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:00.937411 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa8bc84-93ec-422e-aed4-fa4b9ff793cb None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:00.939604 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 160/315] 10.222.0.89 () {56 vars in 1177 bytes} [Thu Jul 28 17:31:00 2022] GET /identity/v3/users/b2c2dfb020434b11af87233d93f83e0f => generated 297 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:00.944862 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f2f141-0a07-49ce-a1ee-5db07a161f38 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:01.055711 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f2f141-0a07-49ce-a1ee-5db07a161f38 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:01.092725 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62a2a7c-0293-4729-b101-1bf8c11bfb4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:01.093732 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62a2a7c-0293-4729-b101-1bf8c11bfb4d None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:01.094220 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62a2a7c-0293-4729-b101-1bf8c11bfb4d None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:01.094751 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62a2a7c-0293-4729-b101-1bf8c11bfb4d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:01.120994 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:01.167640 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62a2a7c-0293-4729-b101-1bf8c11bfb4d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:01.169517 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62a2a7c-0293-4729-b101-1bf8c11bfb4d None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:01.274489 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 156/316] 10.222.0.89 () {58 vars in 1282 bytes} [Thu Jul 28 17:31:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 28 17:31:01.280888 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa8bc84-93ec-422e-aed4-fa4b9ff793cb None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:01.355380 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:01.391468 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa8bc84-93ec-422e-aed4-fa4b9ff793cb None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:01.429883 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7117ed9b-d255-456d-ac9e-79a9c61c1093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:01.430534 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7117ed9b-d255-456d-ac9e-79a9c61c1093 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:01.430814 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7117ed9b-d255-456d-ac9e-79a9c61c1093 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:01.431086 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7117ed9b-d255-456d-ac9e-79a9c61c1093 None admin] PATH_INFO: `/v3/projects/16690c26262a497d9b62ac127ed8bf2e` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:01.432272 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7117ed9b-d255-456d-ac9e-79a9c61c1093 None admin] RBAC: Authorizing `identity:get_project(project_id=16690c26262a497d9b62ac127ed8bf2e)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:01.433743 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7117ed9b-d255-456d-ac9e-79a9c61c1093 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:01.435776 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 161/317] 10.222.0.89 () {56 vars in 1186 bytes} [Thu Jul 28 17:31:01 2022] GET /identity/v3/projects/16690c26262a497d9b62ac127ed8bf2e => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:01.442461 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62a2a7c-0293-4729-b101-1bf8c11bfb4d None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:01.502135 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:01.548704 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62a2a7c-0293-4729-b101-1bf8c11bfb4d None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:01.583617 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d117f735-e914-4b20-ab3e-04191c32df85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:01.584235 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d117f735-e914-4b20-ab3e-04191c32df85 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:01.584514 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d117f735-e914-4b20-ab3e-04191c32df85 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:01.584787 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d117f735-e914-4b20-ab3e-04191c32df85 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:01.585605 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d117f735-e914-4b20-ab3e-04191c32df85 None admin] RBAC: Adding query filter params (role.id=466df921c2db4456b37637c23382d92a, scope.project.id=16690c26262a497d9b62ac127ed8bf2e, user.id=b2c2dfb020434b11af87233d93f83e0f) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:01.585914 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d117f735-e914-4b20-ab3e-04191c32df85 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:01.587505 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d117f735-e914-4b20-ab3e-04191c32df85 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:01.594295 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 157/318] 10.222.0.89 () {56 vars in 1374 bytes} [Thu Jul 28 17:31:01 2022] GET /identity/v3/role_assignments?user.id=b2c2dfb020434b11af87233d93f83e0f&scope.project.id=16690c26262a497d9b62ac127ed8bf2e&role.id=466df921c2db4456b37637c23382d92a => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:02.346065 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ea5411-184d-44d6-bd7d-4cbdae77ad34 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:02.346441 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ea5411-184d-44d6-bd7d-4cbdae77ad34 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:02.346632 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ea5411-184d-44d6-bd7d-4cbdae77ad34 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:02.347997 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 162/319] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:31:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:31:02.349641 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38448bcb-dfbc-43d3-be13-58fecc116c7f None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:02.349928 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38448bcb-dfbc-43d3-be13-58fecc116c7f None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:02.350211 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38448bcb-dfbc-43d3-be13-58fecc116c7f None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:02.351555 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 158/320] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:31:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:31:02.353025 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2a0067-950a-4e3e-a577-ef66a6bc31df None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:02.353346 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2a0067-950a-4e3e-a577-ef66a6bc31df None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:02.353620 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2a0067-950a-4e3e-a577-ef66a6bc31df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:02.356398 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d1b2db-7000-4502-80e2-c93111e25c21 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:02.356680 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d1b2db-7000-4502-80e2-c93111e25c21 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:02.356950 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d1b2db-7000-4502-80e2-c93111e25c21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:02.372156 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-9c2a0067-950a-4e3e-a577-ef66a6bc31df None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:31:02.373378 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-9c2a0067-950a-4e3e-a577-ef66a6bc31df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:31:02.374918 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-24d1b2db-7000-4502-80e2-c93111e25c21 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:31:02.376097 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-24d1b2db-7000-4502-80e2-c93111e25c21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:31:02.444890 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 159/321] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:31:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 17:31:02.455587 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 163/322] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:31:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 17:31:02.469689 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d1b2db-7000-4502-80e2-c93111e25c21 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:02.477583 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2a0067-950a-4e3e-a577-ef66a6bc31df None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:02.524488 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:02.556238 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:02.584763 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d1b2db-7000-4502-80e2-c93111e25c21 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:02.589341 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2a0067-950a-4e3e-a577-ef66a6bc31df None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:02.622987 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7f5051-5b6c-4ab5-bcc7-f4499abb428a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:02.624129 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7f5051-5b6c-4ab5-bcc7-f4499abb428a None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:02.624409 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7f5051-5b6c-4ab5-bcc7-f4499abb428a None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:02.624692 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7f5051-5b6c-4ab5-bcc7-f4499abb428a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:02.626774 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524df659-162d-4efa-9d7b-439eeba5084c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:02.627727 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524df659-162d-4efa-9d7b-439eeba5084c None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:02.628012 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524df659-162d-4efa-9d7b-439eeba5084c None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:02.628282 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524df659-162d-4efa-9d7b-439eeba5084c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:02.635760 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7f5051-5b6c-4ab5-bcc7-f4499abb428a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:31:02.638812 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524df659-162d-4efa-9d7b-439eeba5084c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:31:02.641926 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-8c7f5051-5b6c-4ab5-bcc7-f4499abb428a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 17:31:02.641926 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:02.641926 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:02.641926 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:02.641926 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:02.641926 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:02.641926 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:02.641926 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:02.641926 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:02.641926 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:02.641926 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:02.641926 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:02.641926 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:31:02.641926 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:31:02.641926 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:31:02.641926 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:31:02.642772 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:31:02.642772 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:31:02.642772 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:31:02.642772 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:31:02.642772 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:02.642772 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:02.642772 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:02.642772 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:02.642772 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:02.642772 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:02.642772 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:02.642772 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:02.642772 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:02.642772 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:02.642772 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:02.642772 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:02.643571 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:02.643571 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:02.643571 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:02.643571 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:02.643571 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:31:02.643571 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:31:02.643571 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:31:02.643571 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:31:02.643571 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:31:02.643571 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:31:02.643571 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 17:31:02.643571 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:31:02.644613 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-524df659-162d-4efa-9d7b-439eeba5084c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 17:31:02.644613 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:02.644613 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:02.644613 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:02.644613 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:02.644613 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:02.644613 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:02.644613 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:02.644613 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:02.644613 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:02.644613 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:02.644613 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:02.644613 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:31:02.644613 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:31:02.644613 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:31:02.644613 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:31:02.645647 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:31:02.645647 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:31:02.645647 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:31:02.645647 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:31:02.645647 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:02.645647 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:02.645647 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:02.645647 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:02.645647 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:02.645647 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:02.645647 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:02.645647 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:02.645647 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:02.645647 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:02.645647 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:02.645647 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:02.646465 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:02.646465 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:02.646465 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:02.646465 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:02.646465 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:31:02.646465 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:31:02.646465 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:31:02.646465 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:31:02.646465 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:31:02.646465 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:31:02.646465 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 17:31:02.646465 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:31:02.648097 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 160/323] 10.222.0.89 () {56 vars in 1099 bytes} [Thu Jul 28 17:31:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:31:02.648931 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7f5051-5b6c-4ab5-bcc7-f4499abb428a None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:02.650554 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 164/324] 10.222.0.89 () {56 vars in 1099 bytes} [Thu Jul 28 17:31:02 2022] GET /identity/v3/roles/reader => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:31:02.651402 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524df659-162d-4efa-9d7b-439eeba5084c None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:02.756854 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7f5051-5b6c-4ab5-bcc7-f4499abb428a None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:02.767595 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524df659-162d-4efa-9d7b-439eeba5084c None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:02.770725 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:02.798654 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64db77f-e6ba-4df1-b5e1-a1fff409916c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:02.799269 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64db77f-e6ba-4df1-b5e1-a1fff409916c None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:02.799715 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64db77f-e6ba-4df1-b5e1-a1fff409916c None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:02.799982 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64db77f-e6ba-4df1-b5e1-a1fff409916c None admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:02.800808 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64db77f-e6ba-4df1-b5e1-a1fff409916c None admin] RBAC: Adding query filter params (name=member) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:02.801113 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64db77f-e6ba-4df1-b5e1-a1fff409916c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:02.802524 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64db77f-e6ba-4df1-b5e1-a1fff409916c None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:02.803544 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:02.815714 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9f61cf-0ae9-4bb8-9d8a-7a011f331a04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:02.816345 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9f61cf-0ae9-4bb8-9d8a-7a011f331a04 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:02.816628 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9f61cf-0ae9-4bb8-9d8a-7a011f331a04 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:02.817032 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 161/325] 10.222.0.89 () {56 vars in 1101 bytes} [Thu Jul 28 17:31:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:02.817136 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9f61cf-0ae9-4bb8-9d8a-7a011f331a04 None admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:02.817652 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9f61cf-0ae9-4bb8-9d8a-7a011f331a04 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:02.817991 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9f61cf-0ae9-4bb8-9d8a-7a011f331a04 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:02.819352 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9f61cf-0ae9-4bb8-9d8a-7a011f331a04 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:02.822480 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64db77f-e6ba-4df1-b5e1-a1fff409916c None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:02.838883 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 165/326] 10.222.0.89 () {56 vars in 1101 bytes} [Thu Jul 28 17:31:02 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:02.840179 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9f61cf-0ae9-4bb8-9d8a-7a011f331a04 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:02.935932 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64db77f-e6ba-4df1-b5e1-a1fff409916c None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:02.949276 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9f61cf-0ae9-4bb8-9d8a-7a011f331a04 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:02.970834 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1875c30-1067-47a0-8e60-7af55299a349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:02.971485 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1875c30-1067-47a0-8e60-7af55299a349 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:02.971759 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1875c30-1067-47a0-8e60-7af55299a349 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:02.972034 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1875c30-1067-47a0-8e60-7af55299a349 None admin] PATH_INFO: `/v3/users/c4b04cd90f924f469a7b0b254e5026d8` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:02.973305 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1875c30-1067-47a0-8e60-7af55299a349 None admin] RBAC: Authorizing `identity:get_user(user_id=c4b04cd90f924f469a7b0b254e5026d8)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:02.974866 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1875c30-1067-47a0-8e60-7af55299a349 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:02.976756 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 162/327] 10.222.0.89 () {56 vars in 1177 bytes} [Thu Jul 28 17:31:02 2022] GET /identity/v3/users/c4b04cd90f924f469a7b0b254e5026d8 => generated 293 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:31:02.977979 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0883279b-0eea-4c24-9cc1-13e8ae2d8a8f None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:02.978349 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0883279b-0eea-4c24-9cc1-13e8ae2d8a8f None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:02.978581 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0883279b-0eea-4c24-9cc1-13e8ae2d8a8f None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:02.979888 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 163/328] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:31:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:31:02.985596 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0883279b-0eea-4c24-9cc1-13e8ae2d8a8f None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:02.986633 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46222839-ab0d-4a26-b5a3-2dbb8e848b55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:02.987306 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46222839-ab0d-4a26-b5a3-2dbb8e848b55 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:02.987583 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46222839-ab0d-4a26-b5a3-2dbb8e848b55 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:02.987852 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46222839-ab0d-4a26-b5a3-2dbb8e848b55 None admin] PATH_INFO: `/v3/users/8f014dda3c444b289db91b94c6131013` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:02.989338 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46222839-ab0d-4a26-b5a3-2dbb8e848b55 None admin] RBAC: Authorizing `identity:get_user(user_id=8f014dda3c444b289db91b94c6131013)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:02.990803 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46222839-ab0d-4a26-b5a3-2dbb8e848b55 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:02.992430 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 166/329] 10.222.0.89 () {56 vars in 1177 bytes} [Thu Jul 28 17:31:02 2022] GET /identity/v3/users/8f014dda3c444b289db91b94c6131013 => generated 293 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:31:02.992430 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2db43a9-b085-4af2-a81c-37a02f1cb935 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:02.992430 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2db43a9-b085-4af2-a81c-37a02f1cb935 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:02.992430 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2db43a9-b085-4af2-a81c-37a02f1cb935 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:03.013819 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-a2db43a9-b085-4af2-a81c-37a02f1cb935 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:31:03.015627 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-a2db43a9-b085-4af2-a81c-37a02f1cb935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:31:03.029598 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:03.072010 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:03.084536 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 167/330] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:31:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:31:03.085374 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2db43a9-b085-4af2-a81c-37a02f1cb935 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:03.089801 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0883279b-0eea-4c24-9cc1-13e8ae2d8a8f None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:03.124720 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff80e7f-870a-4f6b-b8d6-e721fe8dfd2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:03.125361 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff80e7f-870a-4f6b-b8d6-e721fe8dfd2a None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:03.125641 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff80e7f-870a-4f6b-b8d6-e721fe8dfd2a None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:03.125922 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff80e7f-870a-4f6b-b8d6-e721fe8dfd2a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:03.126705 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff80e7f-870a-4f6b-b8d6-e721fe8dfd2a None admin] RBAC: Adding query filter params (role.id=11f8731756fe4caca13efa61b71b11c5, user.id=c4b04cd90f924f469a7b0b254e5026d8, scope.system=all) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:03.127013 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff80e7f-870a-4f6b-b8d6-e721fe8dfd2a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:03.128442 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff80e7f-870a-4f6b-b8d6-e721fe8dfd2a None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:03.140600 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 164/331] 10.222.0.89 () {56 vars in 1308 bytes} [Thu Jul 28 17:31:02 2022] GET /identity/v3/role_assignments?user.id=c4b04cd90f924f469a7b0b254e5026d8&scope.system=all&role.id=11f8731756fe4caca13efa61b71b11c5 => generated 510 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:03.148988 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff80e7f-870a-4f6b-b8d6-e721fe8dfd2a None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:03.201027 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2db43a9-b085-4af2-a81c-37a02f1cb935 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:03.240307 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a445c4e-d228-42b6-b681-a48960621ec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:03.241233 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a445c4e-d228-42b6-b681-a48960621ec1 None admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:03.241580 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a445c4e-d228-42b6-b681-a48960621ec1 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:03.241928 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a445c4e-d228-42b6-b681-a48960621ec1 None admin] PATH_INFO: `/v3/system/users/8f014dda3c444b289db91b94c6131013/roles/466df921c2db4456b37637c23382d92a` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:03.243575 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a445c4e-d228-42b6-b681-a48960621ec1 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=8f014dda3c444b289db91b94c6131013, role_id=466df921c2db4456b37637c23382d92a)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:03.245309 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a445c4e-d228-42b6-b681-a48960621ec1 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:03.256532 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 168/332] 10.222.0.89 () {58 vars in 1334 bytes} [Thu Jul 28 17:31:03 2022] PUT /identity/v3/system/users/8f014dda3c444b289db91b94c6131013/roles/466df921c2db4456b37637c23382d92a => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:31:03.262956 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:03.268545 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff80e7f-870a-4f6b-b8d6-e721fe8dfd2a None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:03.315280 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4e60bb-b19a-4e61-b0a5-622f02e4109b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:03.316246 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4e60bb-b19a-4e61-b0a5-622f02e4109b None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:03.316683 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4e60bb-b19a-4e61-b0a5-622f02e4109b None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:03.316958 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4e60bb-b19a-4e61-b0a5-622f02e4109b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:03.318421 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4e60bb-b19a-4e61-b0a5-622f02e4109b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:03.319951 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4e60bb-b19a-4e61-b0a5-622f02e4109b None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:03.321840 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 165/333] 10.222.0.89 () {56 vars in 1108 bytes} [Thu Jul 28 17:31:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:03.328557 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a445c4e-d228-42b6-b681-a48960621ec1 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:03.395836 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:03.444381 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a445c4e-d228-42b6-b681-a48960621ec1 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:03.477954 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cc6ec2-d977-49af-9c48-47ba44089779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:03.478494 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cc6ec2-d977-49af-9c48-47ba44089779 None admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:03.478778 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cc6ec2-d977-49af-9c48-47ba44089779 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:03.479054 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cc6ec2-d977-49af-9c48-47ba44089779 None admin] PATH_INFO: `/v3/groups` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:03.479717 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cc6ec2-d977-49af-9c48-47ba44089779 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:03.481178 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cc6ec2-d977-49af-9c48-47ba44089779 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:03.497353 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 169/334] 10.222.0.89 () {60 vars in 1134 bytes} [Thu Jul 28 17:31:03 2022] POST /identity/v3/groups => generated 226 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:31:05.391987 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fdf236-5015-4200-bbf8-296a691411b0 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:05.392501 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fdf236-5015-4200-bbf8-296a691411b0 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:05.392600 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fdf236-5015-4200-bbf8-296a691411b0 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:05.394383 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 166/335] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:31:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:31:05.399437 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2651d7ca-9e9d-4245-a94e-128a29ecd489 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:05.399838 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2651d7ca-9e9d-4245-a94e-128a29ecd489 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:05.400004 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2651d7ca-9e9d-4245-a94e-128a29ecd489 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:05.423223 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-2651d7ca-9e9d-4245-a94e-128a29ecd489 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:31:05.424438 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-2651d7ca-9e9d-4245-a94e-128a29ecd489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:31:05.511495 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 170/336] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:31:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:31:05.548694 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fdf236-5015-4200-bbf8-296a691411b0 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:05.667999 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fdf236-5015-4200-bbf8-296a691411b0 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:05.707161 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bc5bf6-025f-4780-9049-e6365cce0ea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:05.707818 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bc5bf6-025f-4780-9049-e6365cce0ea1 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:05.708095 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bc5bf6-025f-4780-9049-e6365cce0ea1 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:05.708369 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bc5bf6-025f-4780-9049-e6365cce0ea1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:05.730334 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bc5bf6-025f-4780-9049-e6365cce0ea1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:31:05.736073 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-46bc5bf6-025f-4780-9049-e6365cce0ea1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 17:31:05.736073 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:05.736073 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:05.736073 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:05.736073 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:05.736073 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:05.736073 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:05.736073 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:05.736073 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:05.736073 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:05.736073 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:05.736073 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:05.736073 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:31:05.736073 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:31:05.736073 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:31:05.736073 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:31:05.736967 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:31:05.736967 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:31:05.736967 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:31:05.736967 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:31:05.736967 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:05.736967 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:05.736967 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:05.736967 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:05.736967 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:05.736967 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:05.736967 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:05.736967 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:05.736967 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:05.736967 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:05.736967 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:05.736967 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:05.737806 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:05.737806 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:05.737806 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:05.737806 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:05.737806 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:31:05.737806 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:31:05.737806 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:31:05.737806 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:31:05.737806 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:31:05.737806 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:31:05.737806 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 17:31:05.737806 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:31:05.737806 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 167/337] 10.222.0.89 () {56 vars in 1099 bytes} [Thu Jul 28 17:31:05 2022] GET /identity/v3/roles/reader => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:31:05.743249 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2651d7ca-9e9d-4245-a94e-128a29ecd489 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:05.812405 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:05.857110 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cd3722-93f2-484d-9c27-6a653c4f1721 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:05.857688 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cd3722-93f2-484d-9c27-6a653c4f1721 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:05.858164 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cd3722-93f2-484d-9c27-6a653c4f1721 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:05.859754 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 168/338] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:31:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 17:31:05.872813 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071b56d1-ba11-41b9-968d-c9580bdfb851 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:05.873941 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071b56d1-ba11-41b9-968d-c9580bdfb851 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:05.874826 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071b56d1-ba11-41b9-968d-c9580bdfb851 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:05.907997 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2651d7ca-9e9d-4245-a94e-128a29ecd489 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:05.913699 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-071b56d1-ba11-41b9-968d-c9580bdfb851 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:31:05.915173 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-071b56d1-ba11-41b9-968d-c9580bdfb851 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:31:05.949575 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:05.977937 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0513a848-8d3a-4c27-b897-4b6ba21f2e09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:05.978756 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0513a848-8d3a-4c27-b897-4b6ba21f2e09 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:05.979045 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0513a848-8d3a-4c27-b897-4b6ba21f2e09 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:05.979317 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0513a848-8d3a-4c27-b897-4b6ba21f2e09 None admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:05.980053 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0513a848-8d3a-4c27-b897-4b6ba21f2e09 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:05.981278 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0513a848-8d3a-4c27-b897-4b6ba21f2e09 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:05.982732 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0513a848-8d3a-4c27-b897-4b6ba21f2e09 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:06.010792 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 171/339] 10.222.0.89 () {56 vars in 1101 bytes} [Thu Jul 28 17:31:05 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:06.016634 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0513a848-8d3a-4c27-b897-4b6ba21f2e09 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:06.045066 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 169/340] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:31:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 17:31:06.125690 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071b56d1-ba11-41b9-968d-c9580bdfb851 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:06.213212 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0513a848-8d3a-4c27-b897-4b6ba21f2e09 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:06.226146 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:06.304083 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071b56d1-ba11-41b9-968d-c9580bdfb851 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:06.316947 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bdd6d4-e45d-4a11-8c5a-3e866c6e15c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:06.317631 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bdd6d4-e45d-4a11-8c5a-3e866c6e15c1 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:06.317908 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bdd6d4-e45d-4a11-8c5a-3e866c6e15c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:06.318188 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bdd6d4-e45d-4a11-8c5a-3e866c6e15c1 None admin] PATH_INFO: `/v3/users/8f014dda3c444b289db91b94c6131013` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:06.321447 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bdd6d4-e45d-4a11-8c5a-3e866c6e15c1 None admin] RBAC: Authorizing `identity:get_user(user_id=8f014dda3c444b289db91b94c6131013)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:06.322896 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bdd6d4-e45d-4a11-8c5a-3e866c6e15c1 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:06.324733 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 172/341] 10.222.0.89 () {56 vars in 1177 bytes} [Thu Jul 28 17:31:06 2022] GET /identity/v3/users/8f014dda3c444b289db91b94c6131013 => generated 293 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:31:06.330889 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bdd6d4-e45d-4a11-8c5a-3e866c6e15c1 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:06.412939 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74b7078-4bee-47c8-bfa4-3543343b2511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:06.413635 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74b7078-4bee-47c8-bfa4-3543343b2511 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:06.413899 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74b7078-4bee-47c8-bfa4-3543343b2511 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:06.414172 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74b7078-4bee-47c8-bfa4-3543343b2511 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:06.453277 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74b7078-4bee-47c8-bfa4-3543343b2511 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:31:06.460123 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-a74b7078-4bee-47c8-bfa4-3543343b2511 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 17:31:06.460123 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:06.460123 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:06.460123 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:06.460123 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:06.460123 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:06.460123 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:06.460123 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:06.460123 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:06.460123 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:06.460123 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:06.460123 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:06.460123 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:31:06.460123 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:31:06.460123 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:31:06.460123 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:31:06.461006 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:31:06.461006 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:31:06.461006 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:31:06.461006 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:31:06.461006 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:06.461006 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:06.461006 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:06.461006 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:06.461006 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:06.461006 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:06.461006 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:06.461006 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:06.461006 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:06.461006 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:06.461006 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:06.461006 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:06.462143 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:06.462143 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:06.462143 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:06.462143 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:06.462143 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:31:06.462143 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:31:06.462143 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:31:06.462143 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:31:06.462143 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:31:06.462143 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:31:06.462143 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 17:31:06.462143 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:31:06.462143 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 170/342] 10.222.0.89 () {56 vars in 1099 bytes} [Thu Jul 28 17:31:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:31:06.466112 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74b7078-4bee-47c8-bfa4-3543343b2511 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:06.505936 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bdd6d4-e45d-4a11-8c5a-3e866c6e15c1 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:06.514865 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:06.564367 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea60837-3f83-49db-b968-dd8d1637c258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:06.564988 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea60837-3f83-49db-b968-dd8d1637c258 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:06.570047 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea60837-3f83-49db-b968-dd8d1637c258 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:06.570342 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea60837-3f83-49db-b968-dd8d1637c258 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:06.571154 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea60837-3f83-49db-b968-dd8d1637c258 None admin] RBAC: Adding query filter params (role.id=466df921c2db4456b37637c23382d92a, user.id=8f014dda3c444b289db91b94c6131013, scope.system=all) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:06.571467 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea60837-3f83-49db-b968-dd8d1637c258 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:06.574713 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea60837-3f83-49db-b968-dd8d1637c258 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:06.602738 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 173/343] 10.222.0.89 () {56 vars in 1308 bytes} [Thu Jul 28 17:31:06 2022] GET /identity/v3/role_assignments?user.id=8f014dda3c444b289db91b94c6131013&scope.system=all&role.id=466df921c2db4456b37637c23382d92a => generated 510 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:31:06.662135 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74b7078-4bee-47c8-bfa4-3543343b2511 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:06.715614 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05225458-69bb-4348-a370-312acb05dd37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:06.716313 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05225458-69bb-4348-a370-312acb05dd37 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:06.716541 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05225458-69bb-4348-a370-312acb05dd37 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:06.716819 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05225458-69bb-4348-a370-312acb05dd37 None admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:06.718389 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05225458-69bb-4348-a370-312acb05dd37 None admin] RBAC: Adding query filter params (name=member) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:06.718709 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05225458-69bb-4348-a370-312acb05dd37 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:06.720107 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05225458-69bb-4348-a370-312acb05dd37 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:06.734437 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 171/344] 10.222.0.89 () {56 vars in 1101 bytes} [Thu Jul 28 17:31:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:06.739859 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea60837-3f83-49db-b968-dd8d1637c258 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:06.816889 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:06.922141 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df304c35-51a0-42ad-b0b5-86bbe324360b None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:06.922799 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df304c35-51a0-42ad-b0b5-86bbe324360b None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:06.923446 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df304c35-51a0-42ad-b0b5-86bbe324360b None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:06.926473 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 172/345] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:31:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:31:06.932274 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fd4784-8bc5-4b06-a53f-32c7749a0b62 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:06.932853 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fd4784-8bc5-4b06-a53f-32c7749a0b62 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:06.933365 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fd4784-8bc5-4b06-a53f-32c7749a0b62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:06.940713 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea60837-3f83-49db-b968-dd8d1637c258 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:06.974605 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-48fd4784-8bc5-4b06-a53f-32c7749a0b62 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:31:06.976104 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-48fd4784-8bc5-4b06-a53f-32c7749a0b62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:31:06.988730 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24af1df-61c5-4727-a598-23c4afccd8dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:06.991453 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24af1df-61c5-4727-a598-23c4afccd8dd None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:06.991453 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24af1df-61c5-4727-a598-23c4afccd8dd None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:06.991453 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24af1df-61c5-4727-a598-23c4afccd8dd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:07.084733 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 173/346] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:31:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 17:31:07.086183 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24af1df-61c5-4727-a598-23c4afccd8dd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:07.087660 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24af1df-61c5-4727-a598-23c4afccd8dd None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:07.121740 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:07.126071 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fd4784-8bc5-4b06-a53f-32c7749a0b62 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:07.143489 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:07.202629 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 174/347] 10.222.0.89 () {58 vars in 1282 bytes} [Thu Jul 28 17:31:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 17:31:07.210739 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24af1df-61c5-4727-a598-23c4afccd8dd None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:07.278411 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fd4784-8bc5-4b06-a53f-32c7749a0b62 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:07.330760 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba173145-21e9-4589-a558-ae5258dd4dbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:07.332022 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba173145-21e9-4589-a558-ae5258dd4dbd None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:07.332427 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba173145-21e9-4589-a558-ae5258dd4dbd None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:07.332708 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba173145-21e9-4589-a558-ae5258dd4dbd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:07.342029 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24af1df-61c5-4727-a598-23c4afccd8dd None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:07.346166 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba173145-21e9-4589-a558-ae5258dd4dbd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:31:07.351274 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-ba173145-21e9-4589-a558-ae5258dd4dbd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 17:31:07.351274 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:07.351274 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:07.351274 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:07.351274 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:07.351274 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:07.351274 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:07.351274 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:07.351274 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:07.351274 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:07.351274 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:07.351274 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:07.351274 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:31:07.351274 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:31:07.351274 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:31:07.351274 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:31:07.352314 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:31:07.352314 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:31:07.352314 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:31:07.352314 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:31:07.352314 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:07.352314 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:07.352314 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:07.352314 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:07.352314 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:07.352314 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:07.352314 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:07.352314 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:07.352314 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:07.352314 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:07.352314 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:07.352314 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:07.353754 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:07.353754 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:07.353754 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:07.353754 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:07.353754 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:31:07.353754 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:31:07.353754 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:31:07.353754 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:31:07.353754 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:31:07.353754 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:31:07.353754 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 17:31:07.353754 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:31:07.353754 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 174/348] 10.222.0.89 () {56 vars in 1114 bytes} [Thu Jul 28 17:31:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:31:07.354629 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e2079c-7dda-4270-beac-b7f749b62e8e None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:07.354756 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e2079c-7dda-4270-beac-b7f749b62e8e None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:07.355026 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e2079c-7dda-4270-beac-b7f749b62e8e None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:07.356529 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 175/349] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:31:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:31:07.357582 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1ef2f7-7209-4acc-85d7-cbf947e7c672 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:07.358460 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1ef2f7-7209-4acc-85d7-cbf947e7c672 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:07.358735 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1ef2f7-7209-4acc-85d7-cbf947e7c672 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:07.360428 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 176/350] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:31:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:31:07.362208 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18a86d2-8515-4139-8e5f-eb8afb3a2c88 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:07.362600 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18a86d2-8515-4139-8e5f-eb8afb3a2c88 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:07.362982 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18a86d2-8515-4139-8e5f-eb8afb3a2c88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:07.383796 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:07.384008 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-f18a86d2-8515-4139-8e5f-eb8afb3a2c88 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:31:07.385222 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-f18a86d2-8515-4139-8e5f-eb8afb3a2c88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:31:07.391250 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21afa427-c7fd-4e42-b468-9e5e8fd37be2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:07.392159 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21afa427-c7fd-4e42-b468-9e5e8fd37be2 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:07.392442 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21afa427-c7fd-4e42-b468-9e5e8fd37be2 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:07.392716 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21afa427-c7fd-4e42-b468-9e5e8fd37be2 None admin] PATH_INFO: `/v3/projects/26339f3896c14d148dd3af450ec9d25f` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:07.393974 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21afa427-c7fd-4e42-b468-9e5e8fd37be2 None admin] RBAC: Authorizing `identity:get_project(project_id=26339f3896c14d148dd3af450ec9d25f)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:07.395876 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21afa427-c7fd-4e42-b468-9e5e8fd37be2 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:07.400223 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 175/351] 10.222.0.89 () {56 vars in 1186 bytes} [Thu Jul 28 17:31:07 2022] GET /identity/v3/projects/26339f3896c14d148dd3af450ec9d25f => generated 298 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:31:07.400223 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e197a15c-a3dd-4388-be3c-eb1d71dfc439 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:07.400223 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e197a15c-a3dd-4388-be3c-eb1d71dfc439 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:07.400498 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e197a15c-a3dd-4388-be3c-eb1d71dfc439 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:07.423993 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-e197a15c-a3dd-4388-be3c-eb1d71dfc439 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:31:07.425275 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-e197a15c-a3dd-4388-be3c-eb1d71dfc439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:31:07.476482 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:07.484924 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 177/352] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:31:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:31:07.486002 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18a86d2-8515-4139-8e5f-eb8afb3a2c88 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:07.512866 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 176/353] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:31:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:31:07.512866 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e197a15c-a3dd-4388-be3c-eb1d71dfc439 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:07.603808 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18a86d2-8515-4139-8e5f-eb8afb3a2c88 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:07.637329 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e197a15c-a3dd-4388-be3c-eb1d71dfc439 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:07.656126 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fecdff-ff9d-4930-99e7-312c8da6d698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:07.656760 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fecdff-ff9d-4930-99e7-312c8da6d698 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:07.657041 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fecdff-ff9d-4930-99e7-312c8da6d698 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:07.657339 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fecdff-ff9d-4930-99e7-312c8da6d698 None admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:07.658545 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fecdff-ff9d-4930-99e7-312c8da6d698 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:07.658545 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fecdff-ff9d-4930-99e7-312c8da6d698 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:07.659756 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fecdff-ff9d-4930-99e7-312c8da6d698 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:07.673291 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 178/354] 10.222.0.89 () {56 vars in 1111 bytes} [Thu Jul 28 17:31:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:07.673291 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fecdff-ff9d-4930-99e7-312c8da6d698 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:07.687931 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:07.705742 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0758ce4-d801-4bd3-a75c-84be26ca9d61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:07.706515 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0758ce4-d801-4bd3-a75c-84be26ca9d61 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:07.706941 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0758ce4-d801-4bd3-a75c-84be26ca9d61 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:07.707223 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0758ce4-d801-4bd3-a75c-84be26ca9d61 None admin] PATH_INFO: `/v3/groups/f8ebb3c3767247edaca326e84ef039ec` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:07.716652 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0758ce4-d801-4bd3-a75c-84be26ca9d61 None admin] RBAC: Authorizing `identity:get_group(group_id=f8ebb3c3767247edaca326e84ef039ec)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:07.718690 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0758ce4-d801-4bd3-a75c-84be26ca9d61 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:07.720583 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 177/355] 10.222.0.89 () {56 vars in 1180 bytes} [Thu Jul 28 17:31:07 2022] GET /identity/v3/groups/f8ebb3c3767247edaca326e84ef039ec => generated 226 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:07.721338 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0758ce4-d801-4bd3-a75c-84be26ca9d61 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:07.755193 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:07.808408 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fecdff-ff9d-4930-99e7-312c8da6d698 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:07.851213 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f67ae3-c33f-48aa-b58e-f874aea13f11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:07.852113 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f67ae3-c33f-48aa-b58e-f874aea13f11 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:07.852466 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f67ae3-c33f-48aa-b58e-f874aea13f11 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:07.852811 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f67ae3-c33f-48aa-b58e-f874aea13f11 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:07.857281 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0758ce4-d801-4bd3-a75c-84be26ca9d61 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:07.867657 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f67ae3-c33f-48aa-b58e-f874aea13f11 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:31:07.873879 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-59f67ae3-c33f-48aa-b58e-f874aea13f11 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 17:31:07.873879 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:07.873879 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:07.873879 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:07.873879 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:07.873879 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:07.873879 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:07.873879 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:07.873879 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:07.873879 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:07.873879 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:07.873879 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:07.873879 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:31:07.873879 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:31:07.873879 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:31:07.873879 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:31:07.874816 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:31:07.874816 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:31:07.874816 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:31:07.874816 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:31:07.874816 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:07.874816 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:07.874816 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:07.874816 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:07.874816 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:07.874816 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:07.874816 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:07.874816 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:07.874816 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:07.874816 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:07.874816 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:07.874816 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:07.875623 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:07.875623 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:07.875623 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:07.875623 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:07.875623 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:31:07.875623 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:31:07.875623 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:31:07.875623 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:31:07.875623 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:31:07.875623 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:31:07.875623 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 17:31:07.875623 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:31:07.875623 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 179/356] 10.222.0.89 () {56 vars in 1114 bytes} [Thu Jul 28 17:31:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:31:07.877509 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f67ae3-c33f-48aa-b58e-f874aea13f11 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:07.904725 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fa8227-3b89-47a6-9b42-ed462f79007f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:07.905635 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fa8227-3b89-47a6-9b42-ed462f79007f None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:07.905938 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fa8227-3b89-47a6-9b42-ed462f79007f None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:07.906222 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fa8227-3b89-47a6-9b42-ed462f79007f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:07.918239 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fa8227-3b89-47a6-9b42-ed462f79007f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:31:07.923871 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-d7fa8227-3b89-47a6-9b42-ed462f79007f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 17:31:07.923871 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:07.923871 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:07.923871 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:07.923871 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:07.923871 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:07.923871 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:07.923871 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:07.923871 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:07.923871 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:07.923871 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:07.923871 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:07.923871 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:31:07.923871 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:31:07.923871 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:31:07.923871 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:31:07.924975 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:31:07.924975 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:31:07.924975 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:31:07.924975 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:31:07.924975 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:07.924975 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:07.924975 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:07.924975 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:07.924975 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:07.924975 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:07.924975 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:07.924975 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:07.924975 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:07.924975 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:07.924975 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:07.924975 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:07.925828 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:07.925828 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:07.925828 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:07.925828 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:07.925828 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:31:07.925828 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:31:07.925828 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:31:07.925828 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:31:07.925828 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:31:07.925828 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:31:07.925828 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 17:31:07.925828 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:31:07.925828 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 178/357] 10.222.0.89 () {56 vars in 1099 bytes} [Thu Jul 28 17:31:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:31:07.929112 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fa8227-3b89-47a6-9b42-ed462f79007f None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:07.997462 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:08.011873 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f67ae3-c33f-48aa-b58e-f874aea13f11 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:08.044764 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:08.055500 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fa8227-3b89-47a6-9b42-ed462f79007f None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:08.071699 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27bdb82-9605-4f1b-8351-53ced5622e2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:08.072621 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27bdb82-9605-4f1b-8351-53ced5622e2d None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:08.072907 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27bdb82-9605-4f1b-8351-53ced5622e2d None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:08.073214 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27bdb82-9605-4f1b-8351-53ced5622e2d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:08.101096 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9048f2b-7354-484f-8a5f-164411230b8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:08.102449 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9048f2b-7354-484f-8a5f-164411230b8e None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:08.102449 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9048f2b-7354-484f-8a5f-164411230b8e None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:08.102449 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9048f2b-7354-484f-8a5f-164411230b8e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:08.103072 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9048f2b-7354-484f-8a5f-164411230b8e None admin] RBAC: Adding query filter params (group.id=f8ebb3c3767247edaca326e84ef039ec, role.id=11f8731756fe4caca13efa61b71b11c5, scope.project.id=26339f3896c14d148dd3af450ec9d25f) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:08.103376 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9048f2b-7354-484f-8a5f-164411230b8e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:08.105202 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9048f2b-7354-484f-8a5f-164411230b8e None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:08.112833 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 179/358] 10.222.0.89 () {56 vars in 1376 bytes} [Thu Jul 28 17:31:07 2022] GET /identity/v3/role_assignments?group.id=f8ebb3c3767247edaca326e84ef039ec&scope.project.id=26339f3896c14d148dd3af450ec9d25f&role.id=11f8731756fe4caca13efa61b71b11c5 => generated 260 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:08.113999 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b552dfb6-882b-4491-adb1-65a46a074b1e None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:08.114277 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b552dfb6-882b-4491-adb1-65a46a074b1e None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:08.114550 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b552dfb6-882b-4491-adb1-65a46a074b1e None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:08.115757 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 180/359] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:31:08 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:31:08.116455 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b552dfb6-882b-4491-adb1-65a46a074b1e None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:08.174037 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27bdb82-9605-4f1b-8351-53ced5622e2d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:08.175713 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27bdb82-9605-4f1b-8351-53ced5622e2d None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:08.274211 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b552dfb6-882b-4491-adb1-65a46a074b1e None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:08.296823 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:08.305782 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 180/360] 10.222.0.89 () {58 vars in 1282 bytes} [Thu Jul 28 17:31:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 17:31:08.307031 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27bdb82-9605-4f1b-8351-53ced5622e2d None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:08.319050 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0f68b5-89bb-41b0-870e-7592336e672e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:08.319822 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0f68b5-89bb-41b0-870e-7592336e672e None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:08.320118 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0f68b5-89bb-41b0-870e-7592336e672e None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:08.320371 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0f68b5-89bb-41b0-870e-7592336e672e None admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:08.321094 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0f68b5-89bb-41b0-870e-7592336e672e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:08.321443 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0f68b5-89bb-41b0-870e-7592336e672e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:08.322855 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0f68b5-89bb-41b0-870e-7592336e672e None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:08.332735 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 181/361] 10.222.0.89 () {56 vars in 1111 bytes} [Thu Jul 28 17:31:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:08.333998 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d983d656-e48f-4390-9051-3f875529e393 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:08.334280 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d983d656-e48f-4390-9051-3f875529e393 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:08.334545 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d983d656-e48f-4390-9051-3f875529e393 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:08.337359 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:08.369020 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-d983d656-e48f-4390-9051-3f875529e393 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:31:08.372372 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-d983d656-e48f-4390-9051-3f875529e393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:31:08.441981 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27bdb82-9605-4f1b-8351-53ced5622e2d None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:08.458527 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 182/362] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:31:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:31:08.459377 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d983d656-e48f-4390-9051-3f875529e393 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:08.486575 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19384dc-5563-4bb5-b040-8670dd36b65d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:08.487505 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19384dc-5563-4bb5-b040-8670dd36b65d None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:08.487794 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19384dc-5563-4bb5-b040-8670dd36b65d None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:08.488069 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19384dc-5563-4bb5-b040-8670dd36b65d None admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:08.488795 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19384dc-5563-4bb5-b040-8670dd36b65d None admin] RBAC: Adding query filter params (name=member) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:08.489102 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19384dc-5563-4bb5-b040-8670dd36b65d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:08.490512 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19384dc-5563-4bb5-b040-8670dd36b65d None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:08.502597 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 181/363] 10.222.0.89 () {56 vars in 1101 bytes} [Thu Jul 28 17:31:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:08.504277 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19384dc-5563-4bb5-b040-8670dd36b65d None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:08.602000 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d983d656-e48f-4390-9051-3f875529e393 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:08.616224 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:08.617559 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:08.647286 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19384dc-5563-4bb5-b040-8670dd36b65d None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:08.653347 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7044d18-2122-462e-aa4f-d3a05296b07e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:08.654203 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7044d18-2122-462e-aa4f-d3a05296b07e None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:08.656269 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7044d18-2122-462e-aa4f-d3a05296b07e None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:08.656720 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7044d18-2122-462e-aa4f-d3a05296b07e None admin] PATH_INFO: `/v3/projects/26339f3896c14d148dd3af450ec9d25f` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:08.658471 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7044d18-2122-462e-aa4f-d3a05296b07e None admin] RBAC: Authorizing `identity:get_project(project_id=26339f3896c14d148dd3af450ec9d25f)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:08.661413 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7044d18-2122-462e-aa4f-d3a05296b07e None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:08.663500 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 183/364] 10.222.0.89 () {56 vars in 1186 bytes} [Thu Jul 28 17:31:08 2022] GET /identity/v3/projects/26339f3896c14d148dd3af450ec9d25f => generated 298 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:08.664845 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7044d18-2122-462e-aa4f-d3a05296b07e None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:08.689590 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6a3b14-1e52-410e-87a2-bf6c9b81e36e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:08.690316 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6a3b14-1e52-410e-87a2-bf6c9b81e36e None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:08.690691 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6a3b14-1e52-410e-87a2-bf6c9b81e36e None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:08.691039 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6a3b14-1e52-410e-87a2-bf6c9b81e36e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:08.774944 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6a3b14-1e52-410e-87a2-bf6c9b81e36e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:08.775879 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6a3b14-1e52-410e-87a2-bf6c9b81e36e None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:08.798573 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7044d18-2122-462e-aa4f-d3a05296b07e None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:08.853042 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87f6c46-9fa2-4449-a0fc-7bc3bee98992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:08.853722 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87f6c46-9fa2-4449-a0fc-7bc3bee98992 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:08.854117 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87f6c46-9fa2-4449-a0fc-7bc3bee98992 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:08.854504 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87f6c46-9fa2-4449-a0fc-7bc3bee98992 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:08.867255 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87f6c46-9fa2-4449-a0fc-7bc3bee98992 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:31:08.872480 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-f87f6c46-9fa2-4449-a0fc-7bc3bee98992 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:31:08.872480 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:08.872480 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:08.872480 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:08.872480 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:08.872480 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:08.872480 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:08.872480 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:08.872480 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:08.872480 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:08.872480 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:08.872480 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:08.872480 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:31:08.872480 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:31:08.872480 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:31:08.872480 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:31:08.873548 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:31:08.873548 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:31:08.873548 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:31:08.873548 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:31:08.873548 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:08.873548 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:08.873548 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:08.873548 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:08.873548 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:08.873548 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:08.873548 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:08.873548 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:08.873548 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:08.873548 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:08.873548 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:08.873548 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:08.874394 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:08.874394 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:08.874394 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:08.874394 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:08.874394 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:31:08.874394 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:31:08.874394 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:31:08.874394 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:31:08.874394 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:31:08.874394 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:31:08.874394 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:31:08.874394 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:31:08.875009 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 184/365] 10.222.0.89 () {56 vars in 1096 bytes} [Thu Jul 28 17:31:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:31:08.875930 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87f6c46-9fa2-4449-a0fc-7bc3bee98992 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:08.885456 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:08.916998 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 182/366] 10.222.0.89 () {58 vars in 1282 bytes} [Thu Jul 28 17:31:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 17:31:08.919147 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6a3b14-1e52-410e-87a2-bf6c9b81e36e None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:08.928339 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:09.005292 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87f6c46-9fa2-4449-a0fc-7bc3bee98992 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:09.045179 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6a3b14-1e52-410e-87a2-bf6c9b81e36e None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:09.055853 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a423c4a-94dc-498b-b685-69ba1ddce5cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:09.056484 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a423c4a-94dc-498b-b685-69ba1ddce5cf None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:09.057142 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a423c4a-94dc-498b-b685-69ba1ddce5cf None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:09.057451 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a423c4a-94dc-498b-b685-69ba1ddce5cf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:09.094948 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb276cd-3258-4202-946d-23c382112690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:09.095607 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb276cd-3258-4202-946d-23c382112690 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:09.095889 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb276cd-3258-4202-946d-23c382112690 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:09.096167 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb276cd-3258-4202-946d-23c382112690 None admin] PATH_INFO: `/v3/groups/f8ebb3c3767247edaca326e84ef039ec` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:09.097355 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb276cd-3258-4202-946d-23c382112690 None admin] RBAC: Authorizing `identity:get_group(group_id=f8ebb3c3767247edaca326e84ef039ec)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:09.098857 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb276cd-3258-4202-946d-23c382112690 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:09.100518 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 183/367] 10.222.0.89 () {56 vars in 1180 bytes} [Thu Jul 28 17:31:08 2022] GET /identity/v3/groups/f8ebb3c3767247edaca326e84ef039ec => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:09.101895 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb276cd-3258-4202-946d-23c382112690 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:09.144201 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a423c4a-94dc-498b-b685-69ba1ddce5cf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:09.145798 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a423c4a-94dc-498b-b685-69ba1ddce5cf None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:09.166098 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:09.196381 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:09.233951 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb276cd-3258-4202-946d-23c382112690 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:09.282087 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 185/368] 10.222.0.89 () {58 vars in 1282 bytes} [Thu Jul 28 17:31:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 17:31:09.282960 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a423c4a-94dc-498b-b685-69ba1ddce5cf None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:09.285218 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfefe2ea-07a8-473c-bab7-f5a956d1665f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:09.286864 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfefe2ea-07a8-473c-bab7-f5a956d1665f None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:09.287170 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfefe2ea-07a8-473c-bab7-f5a956d1665f None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:09.287458 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfefe2ea-07a8-473c-bab7-f5a956d1665f None admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:09.288195 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfefe2ea-07a8-473c-bab7-f5a956d1665f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:09.288508 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfefe2ea-07a8-473c-bab7-f5a956d1665f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:09.289920 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfefe2ea-07a8-473c-bab7-f5a956d1665f None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:09.303025 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 184/369] 10.222.0.89 () {56 vars in 1099 bytes} [Thu Jul 28 17:31:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:09.305816 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfefe2ea-07a8-473c-bab7-f5a956d1665f None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:09.423762 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a423c4a-94dc-498b-b685-69ba1ddce5cf None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:09.450803 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfefe2ea-07a8-473c-bab7-f5a956d1665f None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:09.467715 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:09.475566 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb52702-affb-4db0-96ac-2e340deff431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:09.476221 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb52702-affb-4db0-96ac-2e340deff431 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:09.476502 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb52702-affb-4db0-96ac-2e340deff431 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:09.476774 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb52702-affb-4db0-96ac-2e340deff431 None admin] PATH_INFO: `/v3/projects/16690c26262a497d9b62ac127ed8bf2e` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:09.478442 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb52702-affb-4db0-96ac-2e340deff431 None admin] RBAC: Authorizing `identity:get_project(project_id=16690c26262a497d9b62ac127ed8bf2e)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:09.479933 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb52702-affb-4db0-96ac-2e340deff431 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:09.482288 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 186/370] 10.222.0.89 () {56 vars in 1186 bytes} [Thu Jul 28 17:31:09 2022] GET /identity/v3/projects/16690c26262a497d9b62ac127ed8bf2e => generated 302 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:09.482752 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb52702-affb-4db0-96ac-2e340deff431 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:09.495693 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:09.505599 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfb2b7a-dbf6-4961-b625-9bc710572f16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:09.507092 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfb2b7a-dbf6-4961-b625-9bc710572f16 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:09.507541 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfb2b7a-dbf6-4961-b625-9bc710572f16 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:09.507986 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfb2b7a-dbf6-4961-b625-9bc710572f16 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:09.509063 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfb2b7a-dbf6-4961-b625-9bc710572f16 None admin] RBAC: Adding query filter params (group.id=f8ebb3c3767247edaca326e84ef039ec, role.id=895fd016dffb41e7b0205c9a55a4e0f8, scope.project.id=26339f3896c14d148dd3af450ec9d25f) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:09.509889 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfb2b7a-dbf6-4961-b625-9bc710572f16 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:09.511638 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfb2b7a-dbf6-4961-b625-9bc710572f16 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:09.519073 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 185/371] 10.222.0.89 () {56 vars in 1376 bytes} [Thu Jul 28 17:31:09 2022] GET /identity/v3/role_assignments?group.id=f8ebb3c3767247edaca326e84ef039ec&scope.project.id=26339f3896c14d148dd3af450ec9d25f&role.id=895fd016dffb41e7b0205c9a55a4e0f8 => generated 260 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:09.519708 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfb2b7a-dbf6-4961-b625-9bc710572f16 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:09.635380 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb52702-affb-4db0-96ac-2e340deff431 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:09.680959 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfb2b7a-dbf6-4961-b625-9bc710572f16 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:09.681658 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2652f31c-9654-4f2e-9b8c-15851430311f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:09.682598 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2652f31c-9654-4f2e-9b8c-15851430311f None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:09.683077 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2652f31c-9654-4f2e-9b8c-15851430311f None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:09.683546 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2652f31c-9654-4f2e-9b8c-15851430311f None admin] PATH_INFO: `/v3/projects/16690c26262a497d9b62ac127ed8bf2e` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:09.685020 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2652f31c-9654-4f2e-9b8c-15851430311f None admin] RBAC: Authorizing `identity:get_project(project_id=16690c26262a497d9b62ac127ed8bf2e)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:09.687294 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2652f31c-9654-4f2e-9b8c-15851430311f None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:09.690249 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 187/372] 10.222.0.89 () {56 vars in 1186 bytes} [Thu Jul 28 17:31:09 2022] GET /identity/v3/projects/16690c26262a497d9b62ac127ed8bf2e => generated 302 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:09.691411 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2652f31c-9654-4f2e-9b8c-15851430311f None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:09.731177 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d246044-e125-42f6-ac02-fef1c3c667c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:09.732156 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d246044-e125-42f6-ac02-fef1c3c667c7 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:09.732631 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d246044-e125-42f6-ac02-fef1c3c667c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:09.733807 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d246044-e125-42f6-ac02-fef1c3c667c7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:09.798213 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:09.808318 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:09.838665 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2652f31c-9654-4f2e-9b8c-15851430311f None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:09.842061 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d246044-e125-42f6-ac02-fef1c3c667c7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:09.844373 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d246044-e125-42f6-ac02-fef1c3c667c7 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:09.875160 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abed9ee-60b3-4934-ac2d-737b0500a7b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:09.875926 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abed9ee-60b3-4934-ac2d-737b0500a7b1 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:09.876253 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abed9ee-60b3-4934-ac2d-737b0500a7b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:09.876485 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abed9ee-60b3-4934-ac2d-737b0500a7b1 None admin] PATH_INFO: `/v3/groups/f8ebb3c3767247edaca326e84ef039ec` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:09.877748 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abed9ee-60b3-4934-ac2d-737b0500a7b1 None admin] RBAC: Authorizing `identity:get_group(group_id=f8ebb3c3767247edaca326e84ef039ec)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:09.879421 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abed9ee-60b3-4934-ac2d-737b0500a7b1 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:09.881055 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 188/373] 10.222.0.89 () {56 vars in 1180 bytes} [Thu Jul 28 17:31:09 2022] GET /identity/v3/groups/f8ebb3c3767247edaca326e84ef039ec => generated 226 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:09.881932 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abed9ee-60b3-4934-ac2d-737b0500a7b1 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:09.955288 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 186/374] 10.222.0.89 () {58 vars in 1282 bytes} [Thu Jul 28 17:31:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 17:31:09.956390 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d246044-e125-42f6-ac02-fef1c3c667c7 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:09.993341 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abed9ee-60b3-4934-ac2d-737b0500a7b1 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:10.031694 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6d9481-e56b-4840-8190-569148a65bd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:10.032394 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6d9481-e56b-4840-8190-569148a65bd5 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:10.032623 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6d9481-e56b-4840-8190-569148a65bd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:10.032904 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6d9481-e56b-4840-8190-569148a65bd5 None admin] PATH_INFO: `/v3/groups/f8ebb3c3767247edaca326e84ef039ec` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:10.034125 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6d9481-e56b-4840-8190-569148a65bd5 None admin] RBAC: Authorizing `identity:get_group(group_id=f8ebb3c3767247edaca326e84ef039ec)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:10.035584 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6d9481-e56b-4840-8190-569148a65bd5 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:10.041917 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:10.042078 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 189/375] 10.222.0.89 () {56 vars in 1180 bytes} [Thu Jul 28 17:31:09 2022] GET /identity/v3/groups/f8ebb3c3767247edaca326e84ef039ec => generated 226 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:10.042990 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6d9481-e56b-4840-8190-569148a65bd5 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:10.056617 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:10.074304 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d246044-e125-42f6-ac02-fef1c3c667c7 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:10.122619 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600c7662-10b0-48f9-a166-6cba691e3ebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:10.123244 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600c7662-10b0-48f9-a166-6cba691e3ebd None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:10.123704 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600c7662-10b0-48f9-a166-6cba691e3ebd None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:10.124020 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600c7662-10b0-48f9-a166-6cba691e3ebd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:10.125505 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600c7662-10b0-48f9-a166-6cba691e3ebd None admin] RBAC: Adding query filter params (group.id=f8ebb3c3767247edaca326e84ef039ec, role.id=895fd016dffb41e7b0205c9a55a4e0f8, scope.project.id=16690c26262a497d9b62ac127ed8bf2e) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:10.125842 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600c7662-10b0-48f9-a166-6cba691e3ebd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:10.127310 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600c7662-10b0-48f9-a166-6cba691e3ebd None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:10.134668 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 187/376] 10.222.0.89 () {56 vars in 1376 bytes} [Thu Jul 28 17:31:09 2022] GET /identity/v3/role_assignments?group.id=f8ebb3c3767247edaca326e84ef039ec&scope.project.id=16690c26262a497d9b62ac127ed8bf2e&role.id=895fd016dffb41e7b0205c9a55a4e0f8 => generated 260 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:10.135763 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600c7662-10b0-48f9-a166-6cba691e3ebd None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:10.172484 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6d9481-e56b-4840-8190-569148a65bd5 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:10.245959 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c71d05-9044-4ce3-904d-ce900e5046e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:10.246588 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c71d05-9044-4ce3-904d-ce900e5046e8 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:10.246876 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c71d05-9044-4ce3-904d-ce900e5046e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:10.247190 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c71d05-9044-4ce3-904d-ce900e5046e8 None admin] PATH_INFO: `/v3/projects/d68024df9ad444518749a9b417c3b67b` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:10.249510 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c71d05-9044-4ce3-904d-ce900e5046e8 None admin] RBAC: Authorizing `identity:get_project(project_id=d68024df9ad444518749a9b417c3b67b)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:10.251214 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c71d05-9044-4ce3-904d-ce900e5046e8 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:10.253524 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 190/377] 10.222.0.89 () {56 vars in 1186 bytes} [Thu Jul 28 17:31:10 2022] GET /identity/v3/projects/d68024df9ad444518749a9b417c3b67b => generated 344 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:10.260100 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c71d05-9044-4ce3-904d-ce900e5046e8 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:10.334309 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600c7662-10b0-48f9-a166-6cba691e3ebd None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:10.376790 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e86bac-b6fc-4f3e-90f3-60cce80597e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:10.377554 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:10.377683 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e86bac-b6fc-4f3e-90f3-60cce80597e4 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:10.377842 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e86bac-b6fc-4f3e-90f3-60cce80597e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:10.378124 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e86bac-b6fc-4f3e-90f3-60cce80597e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:10.379082 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e86bac-b6fc-4f3e-90f3-60cce80597e4 None admin] RBAC: Adding query filter params (group.id=f8ebb3c3767247edaca326e84ef039ec, role.id=11f8731756fe4caca13efa61b71b11c5, scope.project.id=16690c26262a497d9b62ac127ed8bf2e) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:10.379400 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e86bac-b6fc-4f3e-90f3-60cce80597e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:10.381353 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e86bac-b6fc-4f3e-90f3-60cce80597e4 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:10.388454 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 188/378] 10.222.0.89 () {56 vars in 1376 bytes} [Thu Jul 28 17:31:10 2022] GET /identity/v3/role_assignments?group.id=f8ebb3c3767247edaca326e84ef039ec&scope.project.id=16690c26262a497d9b62ac127ed8bf2e&role.id=11f8731756fe4caca13efa61b71b11c5 => generated 260 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:10.390687 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960f23d7-e48c-42fd-8619-241826c28925 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:10.392348 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960f23d7-e48c-42fd-8619-241826c28925 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:10.392860 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960f23d7-e48c-42fd-8619-241826c28925 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:10.395861 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 189/379] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:31:10 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:31:10.406803 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7e3f20-a0c7-4ed4-9251-39ac23bb6dac None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:10.407131 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7e3f20-a0c7-4ed4-9251-39ac23bb6dac None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:10.407531 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7e3f20-a0c7-4ed4-9251-39ac23bb6dac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:10.411534 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c71d05-9044-4ce3-904d-ce900e5046e8 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:10.429241 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:10.449989 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-7e7e3f20-a0c7-4ed4-9251-39ac23bb6dac None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:31:10.451242 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-7e7e3f20-a0c7-4ed4-9251-39ac23bb6dac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:31:10.501941 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d9cfac-e4b0-4572-b58f-064d02d03940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:10.502522 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d9cfac-e4b0-4572-b58f-064d02d03940 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:10.502807 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d9cfac-e4b0-4572-b58f-064d02d03940 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:10.503080 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d9cfac-e4b0-4572-b58f-064d02d03940 None admin] PATH_INFO: `/v3/groups/c955afc5436941a3b27081a768b9863c` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:10.514677 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d9cfac-e4b0-4572-b58f-064d02d03940 None admin] RBAC: Authorizing `identity:get_group(group_id=c955afc5436941a3b27081a768b9863c)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:10.516182 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d9cfac-e4b0-4572-b58f-064d02d03940 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:10.522918 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 191/380] 10.222.0.89 () {56 vars in 1180 bytes} [Thu Jul 28 17:31:10 2022] GET /identity/v3/groups/c955afc5436941a3b27081a768b9863c => generated 229 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:10.523876 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d9cfac-e4b0-4572-b58f-064d02d03940 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:10.568197 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 190/381] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:31:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:31:10.601802 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7e3f20-a0c7-4ed4-9251-39ac23bb6dac None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:10.672129 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d9cfac-e4b0-4572-b58f-064d02d03940 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:10.710711 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7e3f20-a0c7-4ed4-9251-39ac23bb6dac None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:10.715225 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82345970-7f89-4d0b-b69f-623088c5cad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:10.716204 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82345970-7f89-4d0b-b69f-623088c5cad9 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:10.716681 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82345970-7f89-4d0b-b69f-623088c5cad9 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:10.717692 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82345970-7f89-4d0b-b69f-623088c5cad9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:10.718682 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82345970-7f89-4d0b-b69f-623088c5cad9 None admin] RBAC: Adding query filter params (group.id=c955afc5436941a3b27081a768b9863c, role.id=73e457231a824ea98c28d22610be640c, scope.project.id=d68024df9ad444518749a9b417c3b67b) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:10.719208 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82345970-7f89-4d0b-b69f-623088c5cad9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:10.720957 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82345970-7f89-4d0b-b69f-623088c5cad9 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:10.727486 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 192/382] 10.222.0.89 () {56 vars in 1376 bytes} [Thu Jul 28 17:31:10 2022] GET /identity/v3/role_assignments?group.id=c955afc5436941a3b27081a768b9863c&scope.project.id=d68024df9ad444518749a9b417c3b67b&role.id=73e457231a824ea98c28d22610be640c => generated 260 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:10.751686 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a93be5-7714-4811-b735-24a86d4e2d8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:10.752398 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a93be5-7714-4811-b735-24a86d4e2d8a None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:10.752622 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a93be5-7714-4811-b735-24a86d4e2d8a None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:10.752948 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a93be5-7714-4811-b735-24a86d4e2d8a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:10.755161 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:10.767055 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a93be5-7714-4811-b735-24a86d4e2d8a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:31:10.773790 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-c3a93be5-7714-4811-b735-24a86d4e2d8a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 17:31:10.773790 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:10.773790 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:10.773790 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:10.773790 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:10.773790 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:10.773790 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:10.773790 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:10.773790 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:10.773790 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:10.773790 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:10.773790 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:10.773790 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:31:10.773790 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:31:10.773790 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:31:10.773790 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:31:10.774717 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:31:10.774717 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:31:10.774717 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:31:10.774717 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:31:10.774717 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:10.774717 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:10.774717 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:10.774717 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:10.774717 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:10.774717 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:10.774717 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:10.774717 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:10.774717 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:10.774717 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:10.774717 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:10.774717 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:10.775500 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:10.775500 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:10.775500 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:10.775500 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:10.775500 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:31:10.775500 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:31:10.775500 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:31:10.775500 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:31:10.775500 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:31:10.775500 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:31:10.775500 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 17:31:10.775500 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:31:10.775500 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 191/383] 10.222.0.89 () {56 vars in 1099 bytes} [Thu Jul 28 17:31:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:31:10.783179 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82345970-7f89-4d0b-b69f-623088c5cad9 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:10.802754 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:10.953977 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82345970-7f89-4d0b-b69f-623088c5cad9 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:10.995052 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad7f9ca-51c9-4205-8cd7-ae12e172de81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:10.995669 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad7f9ca-51c9-4205-8cd7-ae12e172de81 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:10.995946 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad7f9ca-51c9-4205-8cd7-ae12e172de81 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:10.996219 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad7f9ca-51c9-4205-8cd7-ae12e172de81 None admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:10.997000 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad7f9ca-51c9-4205-8cd7-ae12e172de81 None admin] RBAC: Adding query filter params (name=member) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:10.997333 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad7f9ca-51c9-4205-8cd7-ae12e172de81 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:10.999871 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad7f9ca-51c9-4205-8cd7-ae12e172de81 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:11.009145 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 193/384] 10.222.0.89 () {56 vars in 1101 bytes} [Thu Jul 28 17:31:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:11.015590 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a93be5-7714-4811-b735-24a86d4e2d8a None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:11.150552 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a93be5-7714-4811-b735-24a86d4e2d8a None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:11.186021 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659232b7-9ba3-4c8d-acdf-635925f02d83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:11.187007 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659232b7-9ba3-4c8d-acdf-635925f02d83 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:11.187509 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659232b7-9ba3-4c8d-acdf-635925f02d83 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:11.187981 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659232b7-9ba3-4c8d-acdf-635925f02d83 None admin] PATH_INFO: `/v3/groups/f8ebb3c3767247edaca326e84ef039ec` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:11.189360 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659232b7-9ba3-4c8d-acdf-635925f02d83 None admin] RBAC: Authorizing `identity:get_group(group_id=f8ebb3c3767247edaca326e84ef039ec)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:11.191001 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659232b7-9ba3-4c8d-acdf-635925f02d83 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:11.193120 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 192/385] 10.222.0.89 () {56 vars in 1180 bytes} [Thu Jul 28 17:31:11 2022] GET /identity/v3/groups/f8ebb3c3767247edaca326e84ef039ec => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:11.198888 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad7f9ca-51c9-4205-8cd7-ae12e172de81 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:11.345989 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:11.421395 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad7f9ca-51c9-4205-8cd7-ae12e172de81 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:11.503549 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5655fd6-b451-4d90-abfc-0863e9f06fb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:11.504216 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5655fd6-b451-4d90-abfc-0863e9f06fb6 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:11.504498 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5655fd6-b451-4d90-abfc-0863e9f06fb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:11.504774 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5655fd6-b451-4d90-abfc-0863e9f06fb6 None admin] PATH_INFO: `/v3/projects/26339f3896c14d148dd3af450ec9d25f` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:11.506014 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5655fd6-b451-4d90-abfc-0863e9f06fb6 None admin] RBAC: Authorizing `identity:get_project(project_id=26339f3896c14d148dd3af450ec9d25f)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:11.507461 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5655fd6-b451-4d90-abfc-0863e9f06fb6 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:11.509139 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 194/386] 10.222.0.89 () {56 vars in 1186 bytes} [Thu Jul 28 17:31:11 2022] GET /identity/v3/projects/26339f3896c14d148dd3af450ec9d25f => generated 298 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:11.515266 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659232b7-9ba3-4c8d-acdf-635925f02d83 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:11.605189 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:11.654556 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659232b7-9ba3-4c8d-acdf-635925f02d83 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:11.702995 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76daf054-9023-4384-8b10-259422e0132a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:11.704020 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76daf054-9023-4384-8b10-259422e0132a None admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:11.704568 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76daf054-9023-4384-8b10-259422e0132a None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:11.705057 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76daf054-9023-4384-8b10-259422e0132a None admin] PATH_INFO: `/v3/projects/26339f3896c14d148dd3af450ec9d25f/groups/f8ebb3c3767247edaca326e84ef039ec/roles/11f8731756fe4caca13efa61b71b11c5` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:11.707231 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76daf054-9023-4384-8b10-259422e0132a None admin] RBAC: Authorizing `identity:create_grant(project_id=26339f3896c14d148dd3af450ec9d25f, group_id=f8ebb3c3767247edaca326e84ef039ec, role_id=11f8731756fe4caca13efa61b71b11c5)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:11.708869 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76daf054-9023-4384-8b10-259422e0132a None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:11.726220 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 193/387] 10.222.0.89 () {58 vars in 1442 bytes} [Thu Jul 28 17:31:11 2022] PUT /identity/v3/projects/26339f3896c14d148dd3af450ec9d25f/groups/f8ebb3c3767247edaca326e84ef039ec/roles/11f8731756fe4caca13efa61b71b11c5 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:31:12.012615 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e874d866-4abf-4eb5-bd7c-5b077d3cd3e3 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:12.013126 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e874d866-4abf-4eb5-bd7c-5b077d3cd3e3 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:12.013225 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e874d866-4abf-4eb5-bd7c-5b077d3cd3e3 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:12.015361 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 195/388] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:31:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:31:12.020781 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c6311a-330a-4f83-880c-5d6cae049a0c None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:12.021074 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c6311a-330a-4f83-880c-5d6cae049a0c None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:12.021381 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c6311a-330a-4f83-880c-5d6cae049a0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:12.041075 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-41c6311a-330a-4f83-880c-5d6cae049a0c None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:31:12.042307 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-41c6311a-330a-4f83-880c-5d6cae049a0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:31:12.121526 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 194/389] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:31:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:31:12.144397 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e874d866-4abf-4eb5-bd7c-5b077d3cd3e3 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:12.277943 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e874d866-4abf-4eb5-bd7c-5b077d3cd3e3 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:12.321299 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938b5e53-d447-49a9-a4bd-8628cf793e05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:12.321945 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938b5e53-d447-49a9-a4bd-8628cf793e05 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:12.322226 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938b5e53-d447-49a9-a4bd-8628cf793e05 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:12.322497 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938b5e53-d447-49a9-a4bd-8628cf793e05 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:12.331719 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:12.341746 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938b5e53-d447-49a9-a4bd-8628cf793e05 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:31:12.347458 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-938b5e53-d447-49a9-a4bd-8628cf793e05 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 17:31:12.347458 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:12.347458 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:12.347458 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:12.347458 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:12.347458 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:12.347458 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:12.347458 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:12.347458 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:12.347458 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:12.347458 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:12.347458 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:12.347458 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:31:12.347458 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:31:12.347458 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:31:12.347458 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:31:12.348387 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:31:12.348387 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:31:12.348387 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:31:12.348387 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:31:12.348387 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:12.348387 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:12.348387 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:12.348387 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:12.348387 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:12.348387 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:12.348387 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:12.348387 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:12.348387 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:12.348387 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:12.348387 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:12.348387 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:12.350231 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:12.350231 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:12.350231 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:12.350231 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:12.350231 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:31:12.350231 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:31:12.350231 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:31:12.350231 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:31:12.350231 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:31:12.350231 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:31:12.350231 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 17:31:12.350231 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:31:12.350231 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 196/390] 10.222.0.89 () {56 vars in 1114 bytes} [Thu Jul 28 17:31:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:31:12.354926 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c6311a-330a-4f83-880c-5d6cae049a0c None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:12.413726 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:12.462166 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c6311a-330a-4f83-880c-5d6cae049a0c None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:12.497905 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e5729d-aad7-4fe6-9bcd-1200ad95e0e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:12.499441 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e5729d-aad7-4fe6-9bcd-1200ad95e0e0 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:12.499869 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e5729d-aad7-4fe6-9bcd-1200ad95e0e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:12.500285 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e5729d-aad7-4fe6-9bcd-1200ad95e0e0 None admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:12.501188 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e5729d-aad7-4fe6-9bcd-1200ad95e0e0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:12.501639 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e5729d-aad7-4fe6-9bcd-1200ad95e0e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:12.503154 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e5729d-aad7-4fe6-9bcd-1200ad95e0e0 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:12.511988 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 195/391] 10.222.0.89 () {56 vars in 1111 bytes} [Thu Jul 28 17:31:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:12.518463 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938b5e53-d447-49a9-a4bd-8628cf793e05 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:12.657833 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938b5e53-d447-49a9-a4bd-8628cf793e05 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:12.702708 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c726586-cd65-4b2d-8ba7-89c2aa3e5050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:12.703633 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c726586-cd65-4b2d-8ba7-89c2aa3e5050 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:12.704061 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c726586-cd65-4b2d-8ba7-89c2aa3e5050 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:12.704475 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c726586-cd65-4b2d-8ba7-89c2aa3e5050 None admin] PATH_INFO: `/v3/groups/f8ebb3c3767247edaca326e84ef039ec` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:12.705842 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c726586-cd65-4b2d-8ba7-89c2aa3e5050 None admin] RBAC: Authorizing `identity:get_group(group_id=f8ebb3c3767247edaca326e84ef039ec)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:12.707477 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c726586-cd65-4b2d-8ba7-89c2aa3e5050 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:12.709491 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 197/392] 10.222.0.89 () {56 vars in 1180 bytes} [Thu Jul 28 17:31:12 2022] GET /identity/v3/groups/f8ebb3c3767247edaca326e84ef039ec => generated 226 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:12.715251 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e5729d-aad7-4fe6-9bcd-1200ad95e0e0 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:12.839671 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e5729d-aad7-4fe6-9bcd-1200ad95e0e0 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:12.873830 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:12.877016 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb345d86-0687-42a4-8329-9c2ddc836432 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:12.877582 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb345d86-0687-42a4-8329-9c2ddc836432 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:12.878042 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb345d86-0687-42a4-8329-9c2ddc836432 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:12.879878 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 198/393] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:31:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:31:12.886576 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba34c3e9-f02b-43a3-afba-6b748c6b9950 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:12.886861 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba34c3e9-f02b-43a3-afba-6b748c6b9950 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:12.887142 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba34c3e9-f02b-43a3-afba-6b748c6b9950 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:12.896101 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5183d60-853e-41ab-963d-af419fad0314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:12.896756 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5183d60-853e-41ab-963d-af419fad0314 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:12.897038 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5183d60-853e-41ab-963d-af419fad0314 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:12.897349 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5183d60-853e-41ab-963d-af419fad0314 None admin] PATH_INFO: `/v3/projects/26339f3896c14d148dd3af450ec9d25f` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:12.899993 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5183d60-853e-41ab-963d-af419fad0314 None admin] RBAC: Authorizing `identity:get_project(project_id=26339f3896c14d148dd3af450ec9d25f)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:12.901464 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5183d60-853e-41ab-963d-af419fad0314 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:12.909929 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 196/394] 10.222.0.89 () {56 vars in 1186 bytes} [Thu Jul 28 17:31:12 2022] GET /identity/v3/projects/26339f3896c14d148dd3af450ec9d25f => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:12.916529 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5183d60-853e-41ab-963d-af419fad0314 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:12.931990 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-ba34c3e9-f02b-43a3-afba-6b748c6b9950 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:31:12.933234 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-ba34c3e9-f02b-43a3-afba-6b748c6b9950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:31:13.027039 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 199/395] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:31:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 17:31:13.027887 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:13.071504 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba34c3e9-f02b-43a3-afba-6b748c6b9950 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:13.119077 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5183d60-853e-41ab-963d-af419fad0314 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:13.164153 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3557264f-e0e4-4fcd-8177-f41914ba6fd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:13.165005 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3557264f-e0e4-4fcd-8177-f41914ba6fd0 None admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:13.165319 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3557264f-e0e4-4fcd-8177-f41914ba6fd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:13.165759 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3557264f-e0e4-4fcd-8177-f41914ba6fd0 None admin] PATH_INFO: `/v3/projects/26339f3896c14d148dd3af450ec9d25f/groups/f8ebb3c3767247edaca326e84ef039ec/roles/895fd016dffb41e7b0205c9a55a4e0f8` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:13.168268 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3557264f-e0e4-4fcd-8177-f41914ba6fd0 None admin] RBAC: Authorizing `identity:create_grant(project_id=26339f3896c14d148dd3af450ec9d25f, group_id=f8ebb3c3767247edaca326e84ef039ec, role_id=895fd016dffb41e7b0205c9a55a4e0f8)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:13.170182 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3557264f-e0e4-4fcd-8177-f41914ba6fd0 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:13.182741 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba34c3e9-f02b-43a3-afba-6b748c6b9950 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:13.197224 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 197/396] 10.222.0.89 () {58 vars in 1442 bytes} [Thu Jul 28 17:31:12 2022] PUT /identity/v3/projects/26339f3896c14d148dd3af450ec9d25f/groups/f8ebb3c3767247edaca326e84ef039ec/roles/895fd016dffb41e7b0205c9a55a4e0f8 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 17:31:13.197224 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52f4260-709d-46a3-bda1-0833d44ad81c None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:13.197224 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52f4260-709d-46a3-bda1-0833d44ad81c None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:13.197224 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52f4260-709d-46a3-bda1-0833d44ad81c None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:13.204827 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 198/397] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:31:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:31:13.206321 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9012815-6bb3-4a37-a7c6-a50e88927618 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:13.206606 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9012815-6bb3-4a37-a7c6-a50e88927618 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:13.206880 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9012815-6bb3-4a37-a7c6-a50e88927618 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:13.259325 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f071398-da76-4c78-8efc-ec8d41fb9ed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:13.259989 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f071398-da76-4c78-8efc-ec8d41fb9ed5 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:13.260265 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f071398-da76-4c78-8efc-ec8d41fb9ed5 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:13.260534 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f071398-da76-4c78-8efc-ec8d41fb9ed5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:13.261786 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-f9012815-6bb3-4a37-a7c6-a50e88927618 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:31:13.263013 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-f9012815-6bb3-4a37-a7c6-a50e88927618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:31:13.287197 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f071398-da76-4c78-8efc-ec8d41fb9ed5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:31:13.293978 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-5f071398-da76-4c78-8efc-ec8d41fb9ed5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 17:31:13.293978 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:13.293978 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:13.293978 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:13.293978 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:13.293978 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:13.293978 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:13.293978 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:13.293978 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:13.293978 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:13.293978 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:13.293978 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:13.293978 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:31:13.293978 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:31:13.293978 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:31:13.293978 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:31:13.295047 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:31:13.295047 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:31:13.295047 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:31:13.295047 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:31:13.295047 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:13.295047 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:13.295047 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:13.295047 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:13.295047 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:13.295047 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:13.295047 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:13.295047 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:13.295047 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:13.295047 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:13.295047 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:13.295047 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:13.296434 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:13.296434 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:13.296434 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:13.296434 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:13.296434 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:31:13.296434 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:31:13.296434 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:31:13.296434 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:31:13.296434 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:31:13.296434 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:31:13.296434 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 17:31:13.296434 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:31:13.296434 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 200/398] 10.222.0.89 () {56 vars in 1114 bytes} [Thu Jul 28 17:31:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:31:13.303759 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f071398-da76-4c78-8efc-ec8d41fb9ed5 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:13.356898 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:13.401637 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:13.408268 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 199/399] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:31:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:31:13.409581 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338d7018-3a5a-400f-8399-e04d77d51e38 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:13.409694 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338d7018-3a5a-400f-8399-e04d77d51e38 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:13.409935 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338d7018-3a5a-400f-8399-e04d77d51e38 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:13.411367 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 200/400] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:31:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:31:13.418706 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b44186f-fe25-432f-bcd6-cc33e8676fe0 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:13.419231 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b44186f-fe25-432f-bcd6-cc33e8676fe0 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:13.419705 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b44186f-fe25-432f-bcd6-cc33e8676fe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:13.445661 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-2b44186f-fe25-432f-bcd6-cc33e8676fe0 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:31:13.447230 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-2b44186f-fe25-432f-bcd6-cc33e8676fe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:31:13.468614 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f071398-da76-4c78-8efc-ec8d41fb9ed5 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:13.517620 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c300b540-8efc-4b5b-9366-ebec48bf721d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:13.519164 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c300b540-8efc-4b5b-9366-ebec48bf721d None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:13.519713 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c300b540-8efc-4b5b-9366-ebec48bf721d None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:13.520331 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c300b540-8efc-4b5b-9366-ebec48bf721d None admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:13.521490 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c300b540-8efc-4b5b-9366-ebec48bf721d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:13.522049 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c300b540-8efc-4b5b-9366-ebec48bf721d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:13.529175 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 201/401] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:31:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 17:31:13.529175 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b44186f-fe25-432f-bcd6-cc33e8676fe0 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:13.532991 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c300b540-8efc-4b5b-9366-ebec48bf721d None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:13.549292 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 201/402] 10.222.0.89 () {56 vars in 1111 bytes} [Thu Jul 28 17:31:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:13.551107 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c300b540-8efc-4b5b-9366-ebec48bf721d None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:13.664505 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b44186f-fe25-432f-bcd6-cc33e8676fe0 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:13.667527 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:13.678184 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c300b540-8efc-4b5b-9366-ebec48bf721d None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:13.697892 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:13.720824 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e38bd0-20f6-40a7-afcd-19d579b5a82f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:13.721568 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e38bd0-20f6-40a7-afcd-19d579b5a82f None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:13.721851 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e38bd0-20f6-40a7-afcd-19d579b5a82f None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:13.722191 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e38bd0-20f6-40a7-afcd-19d579b5a82f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:13.724394 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7872414a-246d-4ab7-ace5-8ecbd18208ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:13.725919 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7872414a-246d-4ab7-ace5-8ecbd18208ee None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:13.726403 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7872414a-246d-4ab7-ace5-8ecbd18208ee None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:13.726878 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7872414a-246d-4ab7-ace5-8ecbd18208ee None admin] PATH_INFO: `/v3/roles/member` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:13.733645 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e38bd0-20f6-40a7-afcd-19d579b5a82f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:31:13.739441 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-f1e38bd0-20f6-40a7-afcd-19d579b5a82f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:31:13.739441 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:13.739441 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:13.739441 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:13.739441 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:13.739441 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:13.739441 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:13.739441 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:13.739441 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:13.739441 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:13.739441 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:13.739441 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:13.739441 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:31:13.739441 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:31:13.739441 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:31:13.739441 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:31:13.740357 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:31:13.740357 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:31:13.740357 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:31:13.740357 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:31:13.740357 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:13.740357 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:13.740357 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:13.740357 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:13.740357 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:13.740357 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:13.740357 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:13.740357 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:13.740357 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:13.740357 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:13.740357 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:13.740357 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:13.741180 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:13.741180 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:13.741180 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:13.741180 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:13.741180 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:31:13.741180 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:31:13.741180 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:31:13.741180 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:31:13.741180 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:31:13.741180 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:31:13.741180 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:31:13.741180 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:31:13.742927 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7872414a-246d-4ab7-ace5-8ecbd18208ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:31:13.745399 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 202/403] 10.222.0.89 () {56 vars in 1096 bytes} [Thu Jul 28 17:31:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:31:13.746234 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e38bd0-20f6-40a7-afcd-19d579b5a82f None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:13.750235 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-7872414a-246d-4ab7-ace5-8ecbd18208ee None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 17:31:13.750235 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:13.750235 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:13.750235 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:13.750235 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:13.750235 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:13.750235 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:13.750235 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:13.750235 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:13.750235 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:13.750235 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:13.750235 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:13.750235 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:31:13.750235 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:31:13.750235 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:31:13.750235 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:31:13.751124 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:31:13.751124 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:31:13.751124 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:31:13.751124 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:31:13.751124 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:13.751124 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:13.751124 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:13.751124 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:13.751124 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:13.751124 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:13.751124 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:13.751124 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:13.751124 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:13.751124 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:13.751124 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:13.751124 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:13.751944 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:13.751944 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:13.751944 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:13.751944 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:13.751944 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:31:13.751944 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:31:13.751944 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:31:13.751944 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:31:13.751944 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:31:13.751944 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:31:13.751944 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 17:31:13.751944 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:31:13.754663 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 202/404] 10.222.0.89 () {56 vars in 1099 bytes} [Thu Jul 28 17:31:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:31:13.755278 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7872414a-246d-4ab7-ace5-8ecbd18208ee None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:13.881606 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e38bd0-20f6-40a7-afcd-19d579b5a82f None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:13.888649 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7872414a-246d-4ab7-ace5-8ecbd18208ee None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:13.916876 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2febbaea-6788-4634-828c-c7b5621c052a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:13.917698 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2febbaea-6788-4634-828c-c7b5621c052a None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:13.917807 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2febbaea-6788-4634-828c-c7b5621c052a None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:13.918090 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2febbaea-6788-4634-828c-c7b5621c052a None admin] PATH_INFO: `/v3/groups/f8ebb3c3767247edaca326e84ef039ec` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:13.919269 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2febbaea-6788-4634-828c-c7b5621c052a None admin] RBAC: Authorizing `identity:get_group(group_id=f8ebb3c3767247edaca326e84ef039ec)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:13.921507 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2febbaea-6788-4634-828c-c7b5621c052a None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:13.923147 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 203/405] 10.222.0.89 () {56 vars in 1180 bytes} [Thu Jul 28 17:31:13 2022] GET /identity/v3/groups/f8ebb3c3767247edaca326e84ef039ec => generated 226 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:13.923877 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2febbaea-6788-4634-828c-c7b5621c052a None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:13.931312 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68e6bb0-476b-4614-9b5e-629acb91e307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:13.932230 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68e6bb0-476b-4614-9b5e-629acb91e307 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:13.933009 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68e6bb0-476b-4614-9b5e-629acb91e307 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:13.933651 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68e6bb0-476b-4614-9b5e-629acb91e307 None admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:13.934639 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68e6bb0-476b-4614-9b5e-629acb91e307 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:13.935146 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68e6bb0-476b-4614-9b5e-629acb91e307 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:13.936900 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68e6bb0-476b-4614-9b5e-629acb91e307 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:13.947093 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 203/406] 10.222.0.89 () {56 vars in 1099 bytes} [Thu Jul 28 17:31:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:13.948480 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68e6bb0-476b-4614-9b5e-629acb91e307 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:13.975125 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:13.980439 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:14.032567 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2febbaea-6788-4634-828c-c7b5621c052a None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:14.067355 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740cc1c2-f0f2-4a78-bf32-24857c2aa213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:14.067972 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740cc1c2-f0f2-4a78-bf32-24857c2aa213 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:14.068253 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740cc1c2-f0f2-4a78-bf32-24857c2aa213 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:14.068528 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740cc1c2-f0f2-4a78-bf32-24857c2aa213 None admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:14.069268 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740cc1c2-f0f2-4a78-bf32-24857c2aa213 None admin] RBAC: Adding query filter params (name=member) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:14.069575 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740cc1c2-f0f2-4a78-bf32-24857c2aa213 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:14.070875 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68e6bb0-476b-4614-9b5e-629acb91e307 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:14.070987 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740cc1c2-f0f2-4a78-bf32-24857c2aa213 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:14.084588 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 204/407] 10.222.0.89 () {56 vars in 1101 bytes} [Thu Jul 28 17:31:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:14.085508 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740cc1c2-f0f2-4a78-bf32-24857c2aa213 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:14.110331 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b66c15-9a7b-4407-b667-5f97bac04366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:14.111491 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b66c15-9a7b-4407-b667-5f97bac04366 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:14.112027 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b66c15-9a7b-4407-b667-5f97bac04366 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:14.112504 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b66c15-9a7b-4407-b667-5f97bac04366 None admin] PATH_INFO: `/v3/projects/16690c26262a497d9b62ac127ed8bf2e` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:14.114058 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b66c15-9a7b-4407-b667-5f97bac04366 None admin] RBAC: Authorizing `identity:get_project(project_id=16690c26262a497d9b62ac127ed8bf2e)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:14.115859 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b66c15-9a7b-4407-b667-5f97bac04366 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:14.118533 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 204/408] 10.222.0.89 () {56 vars in 1186 bytes} [Thu Jul 28 17:31:13 2022] GET /identity/v3/projects/16690c26262a497d9b62ac127ed8bf2e => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:14.119856 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b66c15-9a7b-4407-b667-5f97bac04366 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:14.189656 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740cc1c2-f0f2-4a78-bf32-24857c2aa213 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:14.196769 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:14.223064 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:14.227944 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a01202-8cfc-47af-8966-7c6d3cccb653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:14.228591 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a01202-8cfc-47af-8966-7c6d3cccb653 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:14.228909 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a01202-8cfc-47af-8966-7c6d3cccb653 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:14.229149 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a01202-8cfc-47af-8966-7c6d3cccb653 None admin] PATH_INFO: `/v3/groups/c955afc5436941a3b27081a768b9863c` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:14.230357 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a01202-8cfc-47af-8966-7c6d3cccb653 None admin] RBAC: Authorizing `identity:get_group(group_id=c955afc5436941a3b27081a768b9863c)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:14.231799 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a01202-8cfc-47af-8966-7c6d3cccb653 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:14.233645 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b66c15-9a7b-4407-b667-5f97bac04366 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:14.238948 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 205/409] 10.222.0.89 () {56 vars in 1180 bytes} [Thu Jul 28 17:31:14 2022] GET /identity/v3/groups/c955afc5436941a3b27081a768b9863c => generated 229 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:14.238948 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a01202-8cfc-47af-8966-7c6d3cccb653 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:14.274562 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8398f024-801d-49de-8ade-e2171ba22c85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:14.274802 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8398f024-801d-49de-8ade-e2171ba22c85 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:14.275003 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8398f024-801d-49de-8ade-e2171ba22c85 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:14.275275 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8398f024-801d-49de-8ade-e2171ba22c85 None admin] PATH_INFO: `/v3/groups/f8ebb3c3767247edaca326e84ef039ec` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:14.276526 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8398f024-801d-49de-8ade-e2171ba22c85 None admin] RBAC: Authorizing `identity:get_group(group_id=f8ebb3c3767247edaca326e84ef039ec)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:14.277992 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8398f024-801d-49de-8ade-e2171ba22c85 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:14.284236 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 205/410] 10.222.0.89 () {56 vars in 1180 bytes} [Thu Jul 28 17:31:14 2022] GET /identity/v3/groups/f8ebb3c3767247edaca326e84ef039ec => generated 226 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:14.284236 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc07e4aa-6b02-4c6c-81b0-d8c9c710a157 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:14.284236 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc07e4aa-6b02-4c6c-81b0-d8c9c710a157 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:14.284236 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc07e4aa-6b02-4c6c-81b0-d8c9c710a157 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:14.284236 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 206/411] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:31:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:31:14.284236 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc07e4aa-6b02-4c6c-81b0-d8c9c710a157 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:14.354543 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a01202-8cfc-47af-8966-7c6d3cccb653 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:14.391850 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac88c8c6-8df0-4e81-921a-ddb33ec78568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:14.392531 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac88c8c6-8df0-4e81-921a-ddb33ec78568 None admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:14.392818 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac88c8c6-8df0-4e81-921a-ddb33ec78568 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:14.393093 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac88c8c6-8df0-4e81-921a-ddb33ec78568 None admin] PATH_INFO: `/v3/projects/16690c26262a497d9b62ac127ed8bf2e/groups/f8ebb3c3767247edaca326e84ef039ec/roles/895fd016dffb41e7b0205c9a55a4e0f8` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:14.395447 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac88c8c6-8df0-4e81-921a-ddb33ec78568 None admin] RBAC: Authorizing `identity:create_grant(project_id=16690c26262a497d9b62ac127ed8bf2e, group_id=f8ebb3c3767247edaca326e84ef039ec, role_id=895fd016dffb41e7b0205c9a55a4e0f8)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:14.396950 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac88c8c6-8df0-4e81-921a-ddb33ec78568 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:14.398248 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc07e4aa-6b02-4c6c-81b0-d8c9c710a157 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:14.412705 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 206/412] 10.222.0.89 () {58 vars in 1442 bytes} [Thu Jul 28 17:31:14 2022] PUT /identity/v3/projects/16690c26262a497d9b62ac127ed8bf2e/groups/f8ebb3c3767247edaca326e84ef039ec/roles/895fd016dffb41e7b0205c9a55a4e0f8 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:31:14.413667 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac88c8c6-8df0-4e81-921a-ddb33ec78568 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:14.447284 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1093f6-e5eb-4598-bf94-d2e7f41e04b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:14.448082 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1093f6-e5eb-4598-bf94-d2e7f41e04b9 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:14.448437 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1093f6-e5eb-4598-bf94-d2e7f41e04b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:14.448778 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1093f6-e5eb-4598-bf94-d2e7f41e04b9 None admin] PATH_INFO: `/v3/projects/d68024df9ad444518749a9b417c3b67b` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:14.450265 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1093f6-e5eb-4598-bf94-d2e7f41e04b9 None admin] RBAC: Authorizing `identity:get_project(project_id=d68024df9ad444518749a9b417c3b67b)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:14.452307 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1093f6-e5eb-4598-bf94-d2e7f41e04b9 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:14.459613 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 207/413] 10.222.0.89 () {56 vars in 1186 bytes} [Thu Jul 28 17:31:14 2022] GET /identity/v3/projects/d68024df9ad444518749a9b417c3b67b => generated 344 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:14.461144 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8139bf-addb-450a-92ee-13c908feda5f None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:14.461524 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8139bf-addb-450a-92ee-13c908feda5f None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:14.461867 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8139bf-addb-450a-92ee-13c908feda5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:14.462552 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:14.484120 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-9f8139bf-addb-450a-92ee-13c908feda5f None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:31:14.485524 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-9f8139bf-addb-450a-92ee-13c908feda5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:31:14.516843 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:14.553627 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac88c8c6-8df0-4e81-921a-ddb33ec78568 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:14.592920 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 208/414] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:31:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:31:14.593128 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8139bf-addb-450a-92ee-13c908feda5f None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:14.604842 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476696c7-bbbe-4ed8-ae16-6cd1fa198bee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:14.605585 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476696c7-bbbe-4ed8-ae16-6cd1fa198bee None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:14.605871 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476696c7-bbbe-4ed8-ae16-6cd1fa198bee None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:14.606147 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476696c7-bbbe-4ed8-ae16-6cd1fa198bee None admin] PATH_INFO: `/v3/projects/16690c26262a497d9b62ac127ed8bf2e` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:14.607662 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476696c7-bbbe-4ed8-ae16-6cd1fa198bee None admin] RBAC: Authorizing `identity:get_project(project_id=16690c26262a497d9b62ac127ed8bf2e)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:14.609873 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476696c7-bbbe-4ed8-ae16-6cd1fa198bee None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:14.612158 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 207/415] 10.222.0.89 () {56 vars in 1186 bytes} [Thu Jul 28 17:31:14 2022] GET /identity/v3/projects/16690c26262a497d9b62ac127ed8bf2e => generated 302 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:14.617478 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476696c7-bbbe-4ed8-ae16-6cd1fa198bee None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:14.702047 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8139bf-addb-450a-92ee-13c908feda5f None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:14.740581 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d7026e-928e-439b-ad9b-f63cae18b69c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:14.741264 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d7026e-928e-439b-ad9b-f63cae18b69c None admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:14.741548 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d7026e-928e-439b-ad9b-f63cae18b69c None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:14.741822 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d7026e-928e-439b-ad9b-f63cae18b69c None admin] PATH_INFO: `/v3/projects/d68024df9ad444518749a9b417c3b67b/groups/c955afc5436941a3b27081a768b9863c/roles/73e457231a824ea98c28d22610be640c` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:14.745772 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d7026e-928e-439b-ad9b-f63cae18b69c None admin] RBAC: Authorizing `identity:create_grant(project_id=d68024df9ad444518749a9b417c3b67b, group_id=c955afc5436941a3b27081a768b9863c, role_id=73e457231a824ea98c28d22610be640c)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:14.747214 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d7026e-928e-439b-ad9b-f63cae18b69c None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:14.751575 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476696c7-bbbe-4ed8-ae16-6cd1fa198bee None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:14.753315 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:14.767796 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 209/416] 10.222.0.89 () {58 vars in 1442 bytes} [Thu Jul 28 17:31:14 2022] PUT /identity/v3/projects/d68024df9ad444518749a9b417c3b67b/groups/c955afc5436941a3b27081a768b9863c/roles/73e457231a824ea98c28d22610be640c => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:31:14.769330 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d7026e-928e-439b-ad9b-f63cae18b69c None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:14.821058 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ed1d43-1116-44df-afc8-035116f5302d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:14.821784 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ed1d43-1116-44df-afc8-035116f5302d None admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:14.822115 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ed1d43-1116-44df-afc8-035116f5302d None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:14.822395 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ed1d43-1116-44df-afc8-035116f5302d None admin] PATH_INFO: `/v3/projects/16690c26262a497d9b62ac127ed8bf2e/groups/f8ebb3c3767247edaca326e84ef039ec/roles/11f8731756fe4caca13efa61b71b11c5` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:14.826222 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ed1d43-1116-44df-afc8-035116f5302d None admin] RBAC: Authorizing `identity:create_grant(project_id=16690c26262a497d9b62ac127ed8bf2e, group_id=f8ebb3c3767247edaca326e84ef039ec, role_id=11f8731756fe4caca13efa61b71b11c5)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:14.829705 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ed1d43-1116-44df-afc8-035116f5302d None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:14.837257 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:14.860811 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 208/417] 10.222.0.89 () {58 vars in 1442 bytes} [Thu Jul 28 17:31:14 2022] PUT /identity/v3/projects/16690c26262a497d9b62ac127ed8bf2e/groups/f8ebb3c3767247edaca326e84ef039ec/roles/11f8731756fe4caca13efa61b71b11c5 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 17:31:14.940703 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d7026e-928e-439b-ad9b-f63cae18b69c None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:14.982476 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36968b3-aea7-40fd-a0b2-03f39b71596e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:14.983590 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36968b3-aea7-40fd-a0b2-03f39b71596e None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:14.983867 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36968b3-aea7-40fd-a0b2-03f39b71596e None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:14.984195 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36968b3-aea7-40fd-a0b2-03f39b71596e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:14.996848 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36968b3-aea7-40fd-a0b2-03f39b71596e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:31:15.002829 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-a36968b3-aea7-40fd-a0b2-03f39b71596e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 17:31:15.002829 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:15.002829 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:15.002829 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:15.002829 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:15.002829 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:15.002829 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:15.002829 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:15.002829 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:15.002829 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:15.002829 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:15.002829 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:15.002829 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:31:15.002829 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:31:15.002829 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:31:15.002829 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:31:15.003863 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:31:15.003863 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:31:15.003863 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:31:15.003863 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:31:15.003863 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:15.003863 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:15.003863 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:15.003863 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:15.003863 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:15.003863 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:15.003863 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:15.003863 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:15.003863 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:15.003863 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:15.003863 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:15.003863 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:15.004935 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:15.004935 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:15.004935 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:15.004935 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:15.004935 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:31:15.004935 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:31:15.004935 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:31:15.004935 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:31:15.004935 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:31:15.004935 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:31:15.004935 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 17:31:15.004935 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:31:15.004935 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 210/418] 10.222.0.89 () {56 vars in 1099 bytes} [Thu Jul 28 17:31:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:31:15.009959 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ed1d43-1116-44df-afc8-035116f5302d None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:15.127758 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ed1d43-1116-44df-afc8-035116f5302d None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:15.165695 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050c79ae-3dcc-44f2-a077-f0db247f51e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:15.166321 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050c79ae-3dcc-44f2-a077-f0db247f51e3 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:15.166720 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050c79ae-3dcc-44f2-a077-f0db247f51e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:15.167002 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050c79ae-3dcc-44f2-a077-f0db247f51e3 None admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:15.167729 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050c79ae-3dcc-44f2-a077-f0db247f51e3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:15.168033 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050c79ae-3dcc-44f2-a077-f0db247f51e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:15.169456 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050c79ae-3dcc-44f2-a077-f0db247f51e3 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:15.179609 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 209/419] 10.222.0.89 () {56 vars in 1101 bytes} [Thu Jul 28 17:31:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:31:15.184778 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36968b3-aea7-40fd-a0b2-03f39b71596e None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:15.311626 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:15.345791 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36968b3-aea7-40fd-a0b2-03f39b71596e None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:15.398010 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2415df-0e82-424d-8ffb-80428396f5ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:15.398941 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2415df-0e82-424d-8ffb-80428396f5ea None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:15.399426 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2415df-0e82-424d-8ffb-80428396f5ea None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:15.399898 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2415df-0e82-424d-8ffb-80428396f5ea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:15.429977 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1413131b-1689-42ac-9300-2c4156ec4425 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:15.430558 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1413131b-1689-42ac-9300-2c4156ec4425 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:15.431013 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1413131b-1689-42ac-9300-2c4156ec4425 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:15.432524 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 210/420] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:31:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:31:15.441211 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a2b8eb-ed0c-4084-bd4c-5b96048e38c8 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:15.441211 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a2b8eb-ed0c-4084-bd4c-5b96048e38c8 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:15.441211 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a2b8eb-ed0c-4084-bd4c-5b96048e38c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:15.463223 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-98a2b8eb-ed0c-4084-bd4c-5b96048e38c8 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:31:15.465837 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-98a2b8eb-ed0c-4084-bd4c-5b96048e38c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:31:15.496083 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:15.542592 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2415df-0e82-424d-8ffb-80428396f5ea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:15.543943 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 211/421] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:31:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 17:31:15.544213 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2415df-0e82-424d-8ffb-80428396f5ea None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:15.591939 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a2b8eb-ed0c-4084-bd4c-5b96048e38c8 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:15.707740 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a2b8eb-ed0c-4084-bd4c-5b96048e38c8 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:15.709578 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:15.758244 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 211/422] 10.222.0.89 () {58 vars in 1282 bytes} [Thu Jul 28 17:31:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 574 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 28 17:31:15.760052 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebcd6a9-4d2d-4e05-bd83-3b0eff6172fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:15.760787 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebcd6a9-4d2d-4e05-bd83-3b0eff6172fc None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:15.761073 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebcd6a9-4d2d-4e05-bd83-3b0eff6172fc None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:15.761376 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebcd6a9-4d2d-4e05-bd83-3b0eff6172fc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:15.763851 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2415df-0e82-424d-8ffb-80428396f5ea None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:15.785502 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebcd6a9-4d2d-4e05-bd83-3b0eff6172fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:31:15.792345 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-3ebcd6a9-4d2d-4e05-bd83-3b0eff6172fc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 17:31:15.792345 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:15.792345 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:15.792345 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:15.792345 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:15.792345 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:15.792345 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:15.792345 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:15.792345 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:15.792345 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:15.792345 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:15.792345 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:15.792345 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:31:15.792345 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:31:15.792345 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:31:15.792345 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:31:15.793278 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:31:15.793278 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:31:15.793278 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:31:15.793278 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:31:15.793278 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:15.793278 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:15.793278 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:15.793278 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:15.793278 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:15.793278 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:15.793278 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:15.793278 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:15.793278 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:15.793278 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:15.793278 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:15.793278 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:15.794074 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:15.794074 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:15.794074 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:15.794074 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:15.794074 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:31:15.794074 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:31:15.794074 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:31:15.794074 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:31:15.794074 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:31:15.794074 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:31:15.794074 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 17:31:15.794074 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:31:15.794679 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 212/423] 10.222.0.89 () {56 vars in 1114 bytes} [Thu Jul 28 17:31:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:31:15.802630 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebcd6a9-4d2d-4e05-bd83-3b0eff6172fc None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:15.838931 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:15.921569 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2415df-0e82-424d-8ffb-80428396f5ea None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:15.956906 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebcd6a9-4d2d-4e05-bd83-3b0eff6172fc None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:16.001807 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9396c67-6b9d-44a1-9da4-7dc10b99cd51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:16.002518 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9396c67-6b9d-44a1-9da4-7dc10b99cd51 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:16.002808 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9396c67-6b9d-44a1-9da4-7dc10b99cd51 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:16.003081 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9396c67-6b9d-44a1-9da4-7dc10b99cd51 None admin] PATH_INFO: `/v3/projects/26339f3896c14d148dd3af450ec9d25f` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:16.004521 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9396c67-6b9d-44a1-9da4-7dc10b99cd51 None admin] RBAC: Authorizing `identity:get_project(project_id=26339f3896c14d148dd3af450ec9d25f)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:16.005981 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9396c67-6b9d-44a1-9da4-7dc10b99cd51 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:16.009795 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 212/424] 10.222.0.89 () {56 vars in 1186 bytes} [Thu Jul 28 17:31:15 2022] GET /identity/v3/projects/26339f3896c14d148dd3af450ec9d25f => generated 298 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:16.015253 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9396c67-6b9d-44a1-9da4-7dc10b99cd51 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:16.016529 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d52acd-9755-47c4-872f-437f17c7726b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:16.017168 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d52acd-9755-47c4-872f-437f17c7726b None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:16.017484 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d52acd-9755-47c4-872f-437f17c7726b None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:16.017760 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d52acd-9755-47c4-872f-437f17c7726b None admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:16.020696 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d52acd-9755-47c4-872f-437f17c7726b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:16.021012 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d52acd-9755-47c4-872f-437f17c7726b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:16.022422 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d52acd-9755-47c4-872f-437f17c7726b None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:16.037759 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 213/425] 10.222.0.89 () {56 vars in 1111 bytes} [Thu Jul 28 17:31:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:31:16.043405 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d52acd-9755-47c4-872f-437f17c7726b None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:16.118290 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:16.193505 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:16.230970 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9396c67-6b9d-44a1-9da4-7dc10b99cd51 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:16.244469 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d52acd-9755-47c4-872f-437f17c7726b None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:16.289064 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d328be63-c9cf-4fa5-ad36-65b565be49bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:16.289751 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d328be63-c9cf-4fa5-ad36-65b565be49bb None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:16.290031 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d328be63-c9cf-4fa5-ad36-65b565be49bb None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:16.290304 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d328be63-c9cf-4fa5-ad36-65b565be49bb None admin] PATH_INFO: `/v3/groups/f8ebb3c3767247edaca326e84ef039ec` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:16.295072 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d328be63-c9cf-4fa5-ad36-65b565be49bb None admin] RBAC: Authorizing `identity:get_group(group_id=f8ebb3c3767247edaca326e84ef039ec)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:16.295072 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d328be63-c9cf-4fa5-ad36-65b565be49bb None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:16.295412 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 213/426] 10.222.0.89 () {56 vars in 1180 bytes} [Thu Jul 28 17:31:16 2022] GET /identity/v3/groups/f8ebb3c3767247edaca326e84ef039ec => generated 226 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:31:16.301833 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d328be63-c9cf-4fa5-ad36-65b565be49bb None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:16.337888 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a6145b-ad6f-4654-a11e-f9a450acdc33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:16.338857 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a6145b-ad6f-4654-a11e-f9a450acdc33 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:16.339337 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a6145b-ad6f-4654-a11e-f9a450acdc33 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:16.339796 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a6145b-ad6f-4654-a11e-f9a450acdc33 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:16.468595 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a6145b-ad6f-4654-a11e-f9a450acdc33 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:16.470142 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a6145b-ad6f-4654-a11e-f9a450acdc33 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:16.484958 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d328be63-c9cf-4fa5-ad36-65b565be49bb None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:16.510276 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:16.546460 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6930f62-e333-42df-80af-4a9408b7ad89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:16.547107 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6930f62-e333-42df-80af-4a9408b7ad89 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:16.547390 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6930f62-e333-42df-80af-4a9408b7ad89 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:16.547666 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6930f62-e333-42df-80af-4a9408b7ad89 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:16.548475 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6930f62-e333-42df-80af-4a9408b7ad89 None admin] RBAC: Adding query filter params (group.id=f8ebb3c3767247edaca326e84ef039ec, role.id=11f8731756fe4caca13efa61b71b11c5, scope.project.id=26339f3896c14d148dd3af450ec9d25f) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:16.548790 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6930f62-e333-42df-80af-4a9408b7ad89 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:16.550268 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6930f62-e333-42df-80af-4a9408b7ad89 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:16.556314 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:16.565370 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 214/427] 10.222.0.89 () {56 vars in 1376 bytes} [Thu Jul 28 17:31:16 2022] GET /identity/v3/role_assignments?group.id=f8ebb3c3767247edaca326e84ef039ec&scope.project.id=26339f3896c14d148dd3af450ec9d25f&role.id=11f8731756fe4caca13efa61b71b11c5 => generated 611 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:31:16.584872 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 214/428] 10.222.0.89 () {58 vars in 1282 bytes} [Thu Jul 28 17:31:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 542 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 17:31:16.590397 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6930f62-e333-42df-80af-4a9408b7ad89 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:16.723833 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6930f62-e333-42df-80af-4a9408b7ad89 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:16.760027 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc94cca8-c561-450c-9d20-fc4fb524957e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:16.760649 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc94cca8-c561-450c-9d20-fc4fb524957e None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:16.760937 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc94cca8-c561-450c-9d20-fc4fb524957e None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:16.761237 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc94cca8-c561-450c-9d20-fc4fb524957e None admin] PATH_INFO: `/v3/projects/26339f3896c14d148dd3af450ec9d25f` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:16.762422 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc94cca8-c561-450c-9d20-fc4fb524957e None admin] RBAC: Authorizing `identity:get_project(project_id=26339f3896c14d148dd3af450ec9d25f)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:16.764092 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc94cca8-c561-450c-9d20-fc4fb524957e None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:16.766262 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 215/429] 10.222.0.89 () {56 vars in 1186 bytes} [Thu Jul 28 17:31:16 2022] GET /identity/v3/projects/26339f3896c14d148dd3af450ec9d25f => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:16.771445 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a6145b-ad6f-4654-a11e-f9a450acdc33 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:16.847833 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969e2cc0-33a6-4d3a-a608-65500abd6fad None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:16.848153 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969e2cc0-33a6-4d3a-a608-65500abd6fad None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:16.848387 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969e2cc0-33a6-4d3a-a608-65500abd6fad None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:16.850035 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 216/430] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:31:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:31:16.856277 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26e91ca-6037-4d51-9f88-a9fe73b0685a None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:16.856562 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26e91ca-6037-4d51-9f88-a9fe73b0685a None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:16.856845 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26e91ca-6037-4d51-9f88-a9fe73b0685a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:16.888323 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-c26e91ca-6037-4d51-9f88-a9fe73b0685a None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:31:16.890349 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-c26e91ca-6037-4d51-9f88-a9fe73b0685a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:31:16.923996 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a6145b-ad6f-4654-a11e-f9a450acdc33 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:16.947909 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:16.974027 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba9652d-6421-4e5b-86ba-76d05c2411e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:16.974705 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba9652d-6421-4e5b-86ba-76d05c2411e9 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:16.975092 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba9652d-6421-4e5b-86ba-76d05c2411e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:16.975441 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba9652d-6421-4e5b-86ba-76d05c2411e9 None admin] PATH_INFO: `/v3/groups/f8ebb3c3767247edaca326e84ef039ec` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:16.976705 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba9652d-6421-4e5b-86ba-76d05c2411e9 None admin] RBAC: Authorizing `identity:get_group(group_id=f8ebb3c3767247edaca326e84ef039ec)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:16.978251 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba9652d-6421-4e5b-86ba-76d05c2411e9 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:16.981346 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 215/431] 10.222.0.89 () {56 vars in 1180 bytes} [Thu Jul 28 17:31:16 2022] GET /identity/v3/groups/f8ebb3c3767247edaca326e84ef039ec => generated 226 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:31:16.983399 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 217/432] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:31:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:31:16.987361 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba9652d-6421-4e5b-86ba-76d05c2411e9 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:17.014975 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26e91ca-6037-4d51-9f88-a9fe73b0685a None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:17.057851 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:17.137094 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba9652d-6421-4e5b-86ba-76d05c2411e9 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:17.163358 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26e91ca-6037-4d51-9f88-a9fe73b0685a None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:17.179531 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78879b9-e396-4e77-a92c-b906915ed16a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:17.180521 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78879b9-e396-4e77-a92c-b906915ed16a None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:17.181035 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78879b9-e396-4e77-a92c-b906915ed16a None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:17.181484 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78879b9-e396-4e77-a92c-b906915ed16a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:17.182571 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78879b9-e396-4e77-a92c-b906915ed16a None admin] RBAC: Adding query filter params (group.id=f8ebb3c3767247edaca326e84ef039ec, role.id=895fd016dffb41e7b0205c9a55a4e0f8, scope.project.id=26339f3896c14d148dd3af450ec9d25f) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:17.183065 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78879b9-e396-4e77-a92c-b906915ed16a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:17.184678 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78879b9-e396-4e77-a92c-b906915ed16a None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:17.193660 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 216/433] 10.222.0.89 () {56 vars in 1376 bytes} [Thu Jul 28 17:31:16 2022] GET /identity/v3/role_assignments?group.id=f8ebb3c3767247edaca326e84ef039ec&scope.project.id=26339f3896c14d148dd3af450ec9d25f&role.id=895fd016dffb41e7b0205c9a55a4e0f8 => generated 611 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:17.208288 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168dc433-8a33-4690-a094-c511604c911b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:17.209072 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168dc433-8a33-4690-a094-c511604c911b None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:17.209456 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168dc433-8a33-4690-a094-c511604c911b None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:17.209805 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168dc433-8a33-4690-a094-c511604c911b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:17.225987 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168dc433-8a33-4690-a094-c511604c911b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:31:17.231202 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-168dc433-8a33-4690-a094-c511604c911b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 17:31:17.231202 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:17.231202 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:17.231202 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:17.231202 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:17.231202 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:17.231202 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:17.231202 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:17.231202 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:17.231202 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:17.231202 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:17.231202 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:17.231202 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:31:17.231202 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:31:17.231202 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:31:17.231202 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:31:17.232359 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:31:17.232359 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:31:17.232359 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:31:17.232359 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:31:17.232359 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:17.232359 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:17.232359 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:17.232359 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:17.232359 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:17.232359 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:17.232359 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:17.232359 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:17.232359 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:17.232359 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:17.232359 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:17.232359 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:17.233406 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:17.233406 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:17.233406 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:17.233406 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:17.233406 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:31:17.233406 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:31:17.233406 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:31:17.233406 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:31:17.233406 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:31:17.233406 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:31:17.233406 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 17:31:17.233406 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:31:17.233406 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 218/434] 10.222.0.89 () {56 vars in 1114 bytes} [Thu Jul 28 17:31:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:31:17.240267 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78879b9-e396-4e77-a92c-b906915ed16a None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:17.366246 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78879b9-e396-4e77-a92c-b906915ed16a None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:17.376763 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:17.405874 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9c0022-b852-46e1-9c49-96be3b6ae190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:17.406939 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9c0022-b852-46e1-9c49-96be3b6ae190 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:17.407399 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9c0022-b852-46e1-9c49-96be3b6ae190 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:17.407857 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9c0022-b852-46e1-9c49-96be3b6ae190 None admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:17.408773 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9c0022-b852-46e1-9c49-96be3b6ae190 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:17.409322 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9c0022-b852-46e1-9c49-96be3b6ae190 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:17.410920 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9c0022-b852-46e1-9c49-96be3b6ae190 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:17.421945 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 217/435] 10.222.0.89 () {56 vars in 1111 bytes} [Thu Jul 28 17:31:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:17.427944 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168dc433-8a33-4690-a094-c511604c911b None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:17.461561 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b49b956-f1e7-49f5-b2d0-7294e7369d7d None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:17.461882 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b49b956-f1e7-49f5-b2d0-7294e7369d7d None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:17.462419 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b49b956-f1e7-49f5-b2d0-7294e7369d7d None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:17.463694 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 218/436] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:31:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 17:31:17.466531 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162fdf43-dd9d-46be-b038-59847eb0c960 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:17.466814 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162fdf43-dd9d-46be-b038-59847eb0c960 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:17.467678 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162fdf43-dd9d-46be-b038-59847eb0c960 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:17.469146 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 219/437] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:31:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 17:31:17.471240 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae188cc2-c89f-4c1e-a065-067649ffe9a2 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:17.471526 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae188cc2-c89f-4c1e-a065-067649ffe9a2 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:17.471805 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae188cc2-c89f-4c1e-a065-067649ffe9a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:17.481309 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:17.495135 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-ae188cc2-c89f-4c1e-a065-067649ffe9a2 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:31:17.496356 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-ae188cc2-c89f-4c1e-a065-067649ffe9a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:31:17.563433 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168dc433-8a33-4690-a094-c511604c911b None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:17.573458 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 220/438] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:31:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 17:31:17.576047 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4c9331-849a-4959-84fd-71f0bed4e47b None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:17.576413 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4c9331-849a-4959-84fd-71f0bed4e47b None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:17.576711 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4c9331-849a-4959-84fd-71f0bed4e47b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:17.598961 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-af4c9331-849a-4959-84fd-71f0bed4e47b None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:31:17.600165 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-af4c9331-849a-4959-84fd-71f0bed4e47b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:31:17.606206 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c34156-6958-4121-b137-6c6fa58cf9d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:17.608594 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c34156-6958-4121-b137-6c6fa58cf9d0 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:17.608594 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c34156-6958-4121-b137-6c6fa58cf9d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:17.608594 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c34156-6958-4121-b137-6c6fa58cf9d0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:17.680751 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 221/439] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:31:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:31:17.681669 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4c9331-849a-4959-84fd-71f0bed4e47b None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:17.692349 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c34156-6958-4121-b137-6c6fa58cf9d0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:17.693917 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c34156-6958-4121-b137-6c6fa58cf9d0 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:17.713575 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:17.728424 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:17.786218 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4c9331-849a-4959-84fd-71f0bed4e47b None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:17.796544 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 219/440] 10.222.0.89 () {58 vars in 1282 bytes} [Thu Jul 28 17:31:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 17:31:17.798121 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c34156-6958-4121-b137-6c6fa58cf9d0 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:17.825107 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0984b9d7-eb7f-4d3e-baeb-34fb38044fcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:17.826064 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0984b9d7-eb7f-4d3e-baeb-34fb38044fcc None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:17.826594 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0984b9d7-eb7f-4d3e-baeb-34fb38044fcc None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:17.827012 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0984b9d7-eb7f-4d3e-baeb-34fb38044fcc None admin] PATH_INFO: `/v3/roles/member` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:17.838154 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0984b9d7-eb7f-4d3e-baeb-34fb38044fcc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:31:17.843055 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-0984b9d7-eb7f-4d3e-baeb-34fb38044fcc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 17:31:17.843055 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:17.843055 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:17.843055 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:17.843055 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:17.843055 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:17.843055 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:17.843055 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:17.843055 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:17.843055 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:17.843055 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:17.843055 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:17.843055 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:31:17.843055 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:31:17.843055 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:31:17.843055 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:31:17.844075 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:31:17.844075 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:31:17.844075 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:31:17.844075 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:31:17.844075 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:17.844075 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:17.844075 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:17.844075 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:17.844075 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:17.844075 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:17.844075 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:17.844075 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:17.844075 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:17.844075 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:17.844075 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:17.844075 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:17.844887 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:17.844887 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:17.844887 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:17.844887 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:17.844887 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:31:17.844887 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:31:17.844887 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:31:17.844887 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:31:17.844887 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:31:17.844887 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:31:17.844887 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 17:31:17.844887 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:31:17.844887 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 222/441] 10.222.0.89 () {56 vars in 1099 bytes} [Thu Jul 28 17:31:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:31:17.845595 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0984b9d7-eb7f-4d3e-baeb-34fb38044fcc None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:17.906537 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c34156-6958-4121-b137-6c6fa58cf9d0 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:17.934601 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:17.942718 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13032db-1c6b-4456-a95a-489b406a4153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:17.943361 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13032db-1c6b-4456-a95a-489b406a4153 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:17.943641 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13032db-1c6b-4456-a95a-489b406a4153 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:17.943911 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13032db-1c6b-4456-a95a-489b406a4153 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:17.945237 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0984b9d7-eb7f-4d3e-baeb-34fb38044fcc None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:17.946490 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:17.954778 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13032db-1c6b-4456-a95a-489b406a4153 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:31:17.959681 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-f13032db-1c6b-4456-a95a-489b406a4153 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:31:17.959681 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:17.959681 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:17.959681 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:17.959681 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:17.959681 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:17.959681 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:17.959681 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:17.959681 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:17.959681 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:17.959681 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:17.959681 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:17.959681 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:31:17.959681 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:31:17.959681 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:31:17.959681 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:31:17.960574 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:31:17.960574 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:31:17.960574 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:31:17.960574 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:31:17.960574 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:17.960574 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:17.960574 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:17.960574 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:17.960574 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:17.960574 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:17.960574 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:17.960574 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:17.960574 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:17.960574 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:17.960574 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:17.960574 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:17.961410 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:17.961410 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:17.961410 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:17.961410 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:17.961410 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:31:17.961410 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:31:17.961410 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:31:17.961410 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:31:17.961410 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:31:17.961410 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:31:17.961410 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:31:17.961410 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:31:17.961410 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 220/442] 10.222.0.89 () {56 vars in 1096 bytes} [Thu Jul 28 17:31:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:31:17.962089 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13032db-1c6b-4456-a95a-489b406a4153 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:17.983233 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04984dfa-acf7-4aae-980c-a850ccc2eccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:17.983882 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04984dfa-acf7-4aae-980c-a850ccc2eccf None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:17.984161 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04984dfa-acf7-4aae-980c-a850ccc2eccf None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:17.984435 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04984dfa-acf7-4aae-980c-a850ccc2eccf None admin] PATH_INFO: `/v3/projects/16690c26262a497d9b62ac127ed8bf2e` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:17.985637 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04984dfa-acf7-4aae-980c-a850ccc2eccf None admin] RBAC: Authorizing `identity:get_project(project_id=16690c26262a497d9b62ac127ed8bf2e)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:17.987080 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04984dfa-acf7-4aae-980c-a850ccc2eccf None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:17.988739 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 223/443] 10.222.0.89 () {56 vars in 1186 bytes} [Thu Jul 28 17:31:17 2022] GET /identity/v3/projects/16690c26262a497d9b62ac127ed8bf2e => generated 302 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:17.989533 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04984dfa-acf7-4aae-980c-a850ccc2eccf None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:18.067590 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13032db-1c6b-4456-a95a-489b406a4153 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:18.088423 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04984dfa-acf7-4aae-980c-a850ccc2eccf None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:18.100378 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661fb60c-1b35-46b6-a172-fb296f3b279e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:18.100997 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661fb60c-1b35-46b6-a172-fb296f3b279e None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:18.101315 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661fb60c-1b35-46b6-a172-fb296f3b279e None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:18.101595 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661fb60c-1b35-46b6-a172-fb296f3b279e None admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:18.102311 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661fb60c-1b35-46b6-a172-fb296f3b279e None admin] RBAC: Adding query filter params (name=member) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:18.102615 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661fb60c-1b35-46b6-a172-fb296f3b279e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:18.103990 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661fb60c-1b35-46b6-a172-fb296f3b279e None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:18.112656 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 221/444] 10.222.0.89 () {56 vars in 1101 bytes} [Thu Jul 28 17:31:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:18.113499 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661fb60c-1b35-46b6-a172-fb296f3b279e None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:18.127760 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64008129-fc43-468d-8ed8-3f1c29e67c38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:18.128382 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64008129-fc43-468d-8ed8-3f1c29e67c38 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:18.128664 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64008129-fc43-468d-8ed8-3f1c29e67c38 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:18.128938 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64008129-fc43-468d-8ed8-3f1c29e67c38 None admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:18.129689 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64008129-fc43-468d-8ed8-3f1c29e67c38 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:18.129994 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64008129-fc43-468d-8ed8-3f1c29e67c38 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:18.131358 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64008129-fc43-468d-8ed8-3f1c29e67c38 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:18.139602 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 224/445] 10.222.0.89 () {56 vars in 1099 bytes} [Thu Jul 28 17:31:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:18.140390 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64008129-fc43-468d-8ed8-3f1c29e67c38 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:18.161899 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:18.177937 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:18.224651 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661fb60c-1b35-46b6-a172-fb296f3b279e None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:18.240894 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64008129-fc43-468d-8ed8-3f1c29e67c38 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:18.259505 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e43179-e3ac-41f6-8620-a5070aee0325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:18.260141 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e43179-e3ac-41f6-8620-a5070aee0325 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:18.260419 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e43179-e3ac-41f6-8620-a5070aee0325 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:18.260694 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e43179-e3ac-41f6-8620-a5070aee0325 None admin] PATH_INFO: `/v3/groups/f8ebb3c3767247edaca326e84ef039ec` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:18.261834 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e43179-e3ac-41f6-8620-a5070aee0325 None admin] RBAC: Authorizing `identity:get_group(group_id=f8ebb3c3767247edaca326e84ef039ec)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:18.263251 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e43179-e3ac-41f6-8620-a5070aee0325 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:18.265830 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 222/446] 10.222.0.89 () {56 vars in 1180 bytes} [Thu Jul 28 17:31:18 2022] GET /identity/v3/groups/f8ebb3c3767247edaca326e84ef039ec => generated 226 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:18.266674 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e43179-e3ac-41f6-8620-a5070aee0325 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:18.277590 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091ce6e2-1719-4fcf-a33b-21a11787fb1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:18.278203 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091ce6e2-1719-4fcf-a33b-21a11787fb1f None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:18.278484 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091ce6e2-1719-4fcf-a33b-21a11787fb1f None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:18.278757 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091ce6e2-1719-4fcf-a33b-21a11787fb1f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:18.346446 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091ce6e2-1719-4fcf-a33b-21a11787fb1f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:18.347816 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091ce6e2-1719-4fcf-a33b-21a11787fb1f None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:18.364711 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e43179-e3ac-41f6-8620-a5070aee0325 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:18.367920 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:18.401045 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:18.410475 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443b2412-4949-42c7-bd69-f8411ab288cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:18.411097 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443b2412-4949-42c7-bd69-f8411ab288cb None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:18.411376 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443b2412-4949-42c7-bd69-f8411ab288cb None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:18.411650 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443b2412-4949-42c7-bd69-f8411ab288cb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:18.448816 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 225/447] 10.222.0.89 () {58 vars in 1282 bytes} [Thu Jul 28 17:31:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 17:31:18.449837 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091ce6e2-1719-4fcf-a33b-21a11787fb1f None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:18.480189 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443b2412-4949-42c7-bd69-f8411ab288cb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:18.481686 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443b2412-4949-42c7-bd69-f8411ab288cb None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:18.551830 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091ce6e2-1719-4fcf-a33b-21a11787fb1f None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:18.571884 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:18.583966 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 223/448] 10.222.0.89 () {58 vars in 1282 bytes} [Thu Jul 28 17:31:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 17:31:18.584803 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443b2412-4949-42c7-bd69-f8411ab288cb None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:18.590718 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0692ee02-4a0c-4316-9cdf-3bd5edb16e03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:18.591337 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0692ee02-4a0c-4316-9cdf-3bd5edb16e03 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:18.591619 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0692ee02-4a0c-4316-9cdf-3bd5edb16e03 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:18.591894 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0692ee02-4a0c-4316-9cdf-3bd5edb16e03 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:18.592949 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0692ee02-4a0c-4316-9cdf-3bd5edb16e03 None admin] RBAC: Adding query filter params (group.id=f8ebb3c3767247edaca326e84ef039ec, role.id=895fd016dffb41e7b0205c9a55a4e0f8, scope.project.id=16690c26262a497d9b62ac127ed8bf2e) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:18.593306 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0692ee02-4a0c-4316-9cdf-3bd5edb16e03 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:18.594742 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0692ee02-4a0c-4316-9cdf-3bd5edb16e03 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:18.601744 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 226/449] 10.222.0.89 () {56 vars in 1376 bytes} [Thu Jul 28 17:31:18 2022] GET /identity/v3/role_assignments?group.id=f8ebb3c3767247edaca326e84ef039ec&scope.project.id=16690c26262a497d9b62ac127ed8bf2e&role.id=895fd016dffb41e7b0205c9a55a4e0f8 => generated 611 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:18.602460 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0692ee02-4a0c-4316-9cdf-3bd5edb16e03 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:18.614694 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:18.702225 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443b2412-4949-42c7-bd69-f8411ab288cb None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:18.722354 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0692ee02-4a0c-4316-9cdf-3bd5edb16e03 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:18.741731 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4dcede-6ff2-422d-b410-c3f54b7b3124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:18.742392 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4dcede-6ff2-422d-b410-c3f54b7b3124 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:18.742676 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4dcede-6ff2-422d-b410-c3f54b7b3124 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:18.742949 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4dcede-6ff2-422d-b410-c3f54b7b3124 None admin] PATH_INFO: `/v3/projects/16690c26262a497d9b62ac127ed8bf2e` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:18.744229 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4dcede-6ff2-422d-b410-c3f54b7b3124 None admin] RBAC: Authorizing `identity:get_project(project_id=16690c26262a497d9b62ac127ed8bf2e)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:18.745976 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4dcede-6ff2-422d-b410-c3f54b7b3124 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:18.747930 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 224/450] 10.222.0.89 () {56 vars in 1186 bytes} [Thu Jul 28 17:31:18 2022] GET /identity/v3/projects/16690c26262a497d9b62ac127ed8bf2e => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:18.753431 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4dcede-6ff2-422d-b410-c3f54b7b3124 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:18.760561 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637eca1b-f712-4fb8-8d3e-f91ff2a1d1a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:18.761247 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637eca1b-f712-4fb8-8d3e-f91ff2a1d1a8 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:18.761539 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637eca1b-f712-4fb8-8d3e-f91ff2a1d1a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:18.761816 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637eca1b-f712-4fb8-8d3e-f91ff2a1d1a8 None admin] PATH_INFO: `/v3/projects/d68024df9ad444518749a9b417c3b67b` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:18.764587 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637eca1b-f712-4fb8-8d3e-f91ff2a1d1a8 None admin] RBAC: Authorizing `identity:get_project(project_id=d68024df9ad444518749a9b417c3b67b)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:18.768354 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637eca1b-f712-4fb8-8d3e-f91ff2a1d1a8 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:18.768354 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 227/451] 10.222.0.89 () {56 vars in 1186 bytes} [Thu Jul 28 17:31:18 2022] GET /identity/v3/projects/d68024df9ad444518749a9b417c3b67b => generated 344 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:18.773865 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637eca1b-f712-4fb8-8d3e-f91ff2a1d1a8 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:18.840400 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:18.869035 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:18.877036 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4dcede-6ff2-422d-b410-c3f54b7b3124 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:18.888603 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637eca1b-f712-4fb8-8d3e-f91ff2a1d1a8 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:18.911944 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514649d3-5f14-4985-8281-1488183d64b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:18.912585 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514649d3-5f14-4985-8281-1488183d64b8 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:18.912866 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514649d3-5f14-4985-8281-1488183d64b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:18.913145 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514649d3-5f14-4985-8281-1488183d64b8 None admin] PATH_INFO: `/v3/groups/f8ebb3c3767247edaca326e84ef039ec` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:18.914365 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514649d3-5f14-4985-8281-1488183d64b8 None admin] RBAC: Authorizing `identity:get_group(group_id=f8ebb3c3767247edaca326e84ef039ec)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:18.915809 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514649d3-5f14-4985-8281-1488183d64b8 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:18.917429 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 225/452] 10.222.0.89 () {56 vars in 1180 bytes} [Thu Jul 28 17:31:18 2022] GET /identity/v3/groups/f8ebb3c3767247edaca326e84ef039ec => generated 226 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:31:18.922642 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a5b78a-1615-4a83-8a72-cc2a214d66e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:18.923203 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514649d3-5f14-4985-8281-1488183d64b8 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:18.923300 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a5b78a-1615-4a83-8a72-cc2a214d66e7 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:18.923549 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a5b78a-1615-4a83-8a72-cc2a214d66e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:18.923830 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a5b78a-1615-4a83-8a72-cc2a214d66e7 None admin] PATH_INFO: `/v3/groups/c955afc5436941a3b27081a768b9863c` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:18.924978 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a5b78a-1615-4a83-8a72-cc2a214d66e7 None admin] RBAC: Authorizing `identity:get_group(group_id=c955afc5436941a3b27081a768b9863c)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:18.926443 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a5b78a-1615-4a83-8a72-cc2a214d66e7 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:18.928034 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 228/453] 10.222.0.89 () {56 vars in 1180 bytes} [Thu Jul 28 17:31:18 2022] GET /identity/v3/groups/c955afc5436941a3b27081a768b9863c => generated 229 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:31:18.933604 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a5b78a-1615-4a83-8a72-cc2a214d66e7 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:19.026030 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514649d3-5f14-4985-8281-1488183d64b8 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:19.034372 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a5b78a-1615-4a83-8a72-cc2a214d66e7 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:19.053890 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:19.060070 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bc8975-e828-4adc-ad47-6d395d5b9c94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:19.061208 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bc8975-e828-4adc-ad47-6d395d5b9c94 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:19.061846 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bc8975-e828-4adc-ad47-6d395d5b9c94 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:19.062110 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bc8975-e828-4adc-ad47-6d395d5b9c94 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:19.062897 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bc8975-e828-4adc-ad47-6d395d5b9c94 None admin] RBAC: Adding query filter params (group.id=f8ebb3c3767247edaca326e84ef039ec, role.id=11f8731756fe4caca13efa61b71b11c5, scope.project.id=16690c26262a497d9b62ac127ed8bf2e) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:19.063200 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bc8975-e828-4adc-ad47-6d395d5b9c94 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:19.064631 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bc8975-e828-4adc-ad47-6d395d5b9c94 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:19.070151 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6de216-bd52-4782-b714-1ad9dd3eb054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:19.070481 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 226/454] 10.222.0.89 () {56 vars in 1376 bytes} [Thu Jul 28 17:31:18 2022] GET /identity/v3/role_assignments?group.id=f8ebb3c3767247edaca326e84ef039ec&scope.project.id=16690c26262a497d9b62ac127ed8bf2e&role.id=11f8731756fe4caca13efa61b71b11c5 => generated 611 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:31:19.070839 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6de216-bd52-4782-b714-1ad9dd3eb054 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:19.071123 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6de216-bd52-4782-b714-1ad9dd3eb054 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:19.071394 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6de216-bd52-4782-b714-1ad9dd3eb054 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:19.072238 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6de216-bd52-4782-b714-1ad9dd3eb054 None admin] RBAC: Adding query filter params (group.id=c955afc5436941a3b27081a768b9863c, role.id=73e457231a824ea98c28d22610be640c, scope.project.id=d68024df9ad444518749a9b417c3b67b) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:19.072548 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6de216-bd52-4782-b714-1ad9dd3eb054 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:19.074074 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6de216-bd52-4782-b714-1ad9dd3eb054 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:19.081818 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 229/455] 10.222.0.89 () {56 vars in 1376 bytes} [Thu Jul 28 17:31:18 2022] GET /identity/v3/role_assignments?group.id=c955afc5436941a3b27081a768b9863c&scope.project.id=d68024df9ad444518749a9b417c3b67b&role.id=73e457231a824ea98c28d22610be640c => generated 611 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:31:21.799063 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23620f3-41ba-44f6-8987-279e44571fb2 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:21.799619 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23620f3-41ba-44f6-8987-279e44571fb2 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:21.799619 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23620f3-41ba-44f6-8987-279e44571fb2 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:21.801135 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 227/456] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:31:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:31:21.806084 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c60fef-67c2-4ae1-8483-d0633e5b66ee None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:21.806936 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c60fef-67c2-4ae1-8483-d0633e5b66ee None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:21.807482 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c60fef-67c2-4ae1-8483-d0633e5b66ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:21.831059 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-42c60fef-67c2-4ae1-8483-d0633e5b66ee None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:31:21.832470 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-42c60fef-67c2-4ae1-8483-d0633e5b66ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:31:21.921010 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 230/457] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:31:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:31:21.957612 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23620f3-41ba-44f6-8987-279e44571fb2 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:21.992352 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:22.048325 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67435c74-6030-45f7-9880-a67e81b05402 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:22.048652 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67435c74-6030-45f7-9880-a67e81b05402 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:22.048874 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67435c74-6030-45f7-9880-a67e81b05402 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:22.050362 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 231/458] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:31:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 17:31:22.066722 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b86ba5-0a99-4717-99f0-1604e09f7242 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:22.067250 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b86ba5-0a99-4717-99f0-1604e09f7242 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:22.067670 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b86ba5-0a99-4717-99f0-1604e09f7242 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:22.098318 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23620f3-41ba-44f6-8987-279e44571fb2 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:22.104531 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-79b86ba5-0a99-4717-99f0-1604e09f7242 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:31:22.107737 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-79b86ba5-0a99-4717-99f0-1604e09f7242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:31:22.138707 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bf5acc-d19f-4e27-af38-95540e5fff4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:22.139475 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bf5acc-d19f-4e27-af38-95540e5fff4e None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:22.140043 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bf5acc-d19f-4e27-af38-95540e5fff4e None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:22.140329 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bf5acc-d19f-4e27-af38-95540e5fff4e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:22.153238 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bf5acc-d19f-4e27-af38-95540e5fff4e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:22.155782 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bf5acc-d19f-4e27-af38-95540e5fff4e None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:22.175886 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-28bf5acc-d19f-4e27-af38-95540e5fff4e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:31:22.175886 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:22.175886 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:22.175886 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:22.175886 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:22.175886 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:22.175886 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:22.175886 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:22.175886 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:22.175886 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:22.175886 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:22.175886 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:22.175886 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:31:22.175886 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:31:22.175886 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:31:22.176976 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:31:22.176976 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:22.176976 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:22.176976 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:22.176976 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:22.176976 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:22.176976 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:22.176976 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:22.176976 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:22.176976 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:22.176976 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:22.176976 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:22.176976 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:22.176976 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:22.176976 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:22.176976 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:22.176976 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:22.177898 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:31:22.177898 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:31:22.177898 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:31:22.177898 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:31:22.177898 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 228/459] 10.222.0.89 () {56 vars in 1108 bytes} [Thu Jul 28 17:31:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:31:22.178565 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f143d87-5587-4d8a-adf8-ac6ab16641c9 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:22.178854 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f143d87-5587-4d8a-adf8-ac6ab16641c9 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:22.179132 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f143d87-5587-4d8a-adf8-ac6ab16641c9 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:22.180660 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 229/460] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:31:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:31:22.187635 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f143d87-5587-4d8a-adf8-ac6ab16641c9 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:22.189354 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 232/461] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:31:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:31:22.190806 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1c67b1-dd3f-409b-87ce-d19ede326a30 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:22.191240 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1c67b1-dd3f-409b-87ce-d19ede326a30 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:22.191516 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1c67b1-dd3f-409b-87ce-d19ede326a30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:22.210182 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-0d1c67b1-dd3f-409b-87ce-d19ede326a30 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:31:22.211411 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-0d1c67b1-dd3f-409b-87ce-d19ede326a30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:31:22.270015 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:22.287758 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:22.312468 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 233/462] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:31:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:31:22.313716 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1c67b1-dd3f-409b-87ce-d19ede326a30 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:22.318326 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f143d87-5587-4d8a-adf8-ac6ab16641c9 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:22.361520 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc586b58-261a-4e65-89b5-f9c647443e36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:22.362414 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc586b58-261a-4e65-89b5-f9c647443e36 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:22.362826 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc586b58-261a-4e65-89b5-f9c647443e36 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:22.363231 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc586b58-261a-4e65-89b5-f9c647443e36 None admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:22.364072 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc586b58-261a-4e65-89b5-f9c647443e36 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:22.364506 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc586b58-261a-4e65-89b5-f9c647443e36 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:22.366072 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc586b58-261a-4e65-89b5-f9c647443e36 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:22.387569 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 230/463] 10.222.0.89 () {56 vars in 1109 bytes} [Thu Jul 28 17:31:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:22.389604 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc586b58-261a-4e65-89b5-f9c647443e36 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:22.458700 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1c67b1-dd3f-409b-87ce-d19ede326a30 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:22.497030 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc586b58-261a-4e65-89b5-f9c647443e36 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:22.498321 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c287c5-7337-47c2-b374-312665f2da56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:22.499517 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c287c5-7337-47c2-b374-312665f2da56 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:22.499943 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c287c5-7337-47c2-b374-312665f2da56 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:22.500349 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c287c5-7337-47c2-b374-312665f2da56 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:22.511719 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c287c5-7337-47c2-b374-312665f2da56 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:22.513456 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c287c5-7337-47c2-b374-312665f2da56 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:22.531871 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-d6c287c5-7337-47c2-b374-312665f2da56 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:31:22.531871 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:22.531871 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:22.531871 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:22.531871 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:22.531871 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:22.531871 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:22.531871 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:22.531871 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:22.531871 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:22.531871 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:22.531871 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:22.531871 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:31:22.531871 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:31:22.531871 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:31:22.532915 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:31:22.532915 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:22.532915 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:22.532915 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:22.532915 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:22.532915 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:22.532915 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:22.532915 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:22.532915 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:22.532915 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:22.532915 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:22.532915 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:22.532915 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:22.532915 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:22.532915 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:22.532915 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:22.532915 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:22.533794 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:31:22.533794 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:31:22.533794 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:31:22.533794 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:31:22.533794 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 234/464] 10.222.0.89 () {56 vars in 1108 bytes} [Thu Jul 28 17:31:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:31:22.535297 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c287c5-7337-47c2-b374-312665f2da56 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:22.540619 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a2913c-fc08-4f7f-bddb-95bc00617c51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:22.541325 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a2913c-fc08-4f7f-bddb-95bc00617c51 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:22.541611 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a2913c-fc08-4f7f-bddb-95bc00617c51 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:22.541884 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a2913c-fc08-4f7f-bddb-95bc00617c51 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:22.550861 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:22.556457 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a2913c-fc08-4f7f-bddb-95bc00617c51 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:22.558194 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a2913c-fc08-4f7f-bddb-95bc00617c51 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:22.571100 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:22.578606 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-85a2913c-fc08-4f7f-bddb-95bc00617c51 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:31:22.578606 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:22.578606 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:22.578606 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:22.578606 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:22.578606 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:22.578606 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:22.578606 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:22.578606 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:22.578606 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:22.578606 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:22.578606 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:22.578606 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:31:22.578606 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:31:22.578606 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:31:22.579671 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:31:22.579671 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:22.579671 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:22.579671 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:22.579671 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:22.579671 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:22.579671 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:22.579671 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:22.579671 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:22.579671 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:22.579671 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:22.579671 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:22.579671 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:22.579671 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:22.579671 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:22.579671 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:22.579671 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:22.580731 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:31:22.580731 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:31:22.580731 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:31:22.580731 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:31:22.587092 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 231/465] 10.222.0.89 () {56 vars in 1108 bytes} [Thu Jul 28 17:31:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:31:22.588351 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a2913c-fc08-4f7f-bddb-95bc00617c51 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:22.656231 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c287c5-7337-47c2-b374-312665f2da56 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:22.696635 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfe3c85-f925-4367-b3a7-38f119355e02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:22.697440 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfe3c85-f925-4367-b3a7-38f119355e02 None admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:22.697929 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfe3c85-f925-4367-b3a7-38f119355e02 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:22.698282 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfe3c85-f925-4367-b3a7-38f119355e02 None admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:22.699041 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfe3c85-f925-4367-b3a7-38f119355e02 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:22.700622 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfe3c85-f925-4367-b3a7-38f119355e02 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:22.714852 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a2913c-fc08-4f7f-bddb-95bc00617c51 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:22.725932 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 235/466] 10.222.0.89 () {60 vars in 1132 bytes} [Thu Jul 28 17:31:22 2022] POST /identity/v3/users => generated 246 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:31:22.727319 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfe3c85-f925-4367-b3a7-38f119355e02 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:22.768030 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f80542-2542-48c8-b928-258e902e47ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:22.768682 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f80542-2542-48c8-b928-258e902e47ba None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:22.768965 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f80542-2542-48c8-b928-258e902e47ba None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:22.769286 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f80542-2542-48c8-b928-258e902e47ba None admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:22.770006 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f80542-2542-48c8-b928-258e902e47ba None admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:22.770313 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f80542-2542-48c8-b928-258e902e47ba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:22.771728 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f80542-2542-48c8-b928-258e902e47ba None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:22.790113 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 232/467] 10.222.0.89 () {56 vars in 1109 bytes} [Thu Jul 28 17:31:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:22.791590 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fdd219-10a7-42fa-a2c2-d37146d1d4dd None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:22.791881 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fdd219-10a7-42fa-a2c2-d37146d1d4dd None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:22.792160 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fdd219-10a7-42fa-a2c2-d37146d1d4dd None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:22.794038 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 233/468] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:31:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:31:22.794851 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fdd219-10a7-42fa-a2c2-d37146d1d4dd None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:22.843440 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:22.869588 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfe3c85-f925-4367-b3a7-38f119355e02 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:22.876537 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:22.916518 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f28f784-a075-4ca9-8a31-7257d5033afa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:22.917821 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f28f784-a075-4ca9-8a31-7257d5033afa None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:22.918351 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f28f784-a075-4ca9-8a31-7257d5033afa None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:22.918880 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f28f784-a075-4ca9-8a31-7257d5033afa None admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:22.919786 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f28f784-a075-4ca9-8a31-7257d5033afa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:22.920419 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f28f784-a075-4ca9-8a31-7257d5033afa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:22.922222 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f28f784-a075-4ca9-8a31-7257d5033afa None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:22.935492 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 236/469] 10.222.0.89 () {56 vars in 1109 bytes} [Thu Jul 28 17:31:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:22.937075 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fdd219-10a7-42fa-a2c2-d37146d1d4dd None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:22.937941 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bf1a03-3c52-4cb2-b6e6-630a29a7afae None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:22.938374 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bf1a03-3c52-4cb2-b6e6-630a29a7afae None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:22.938849 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bf1a03-3c52-4cb2-b6e6-630a29a7afae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:22.957443 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-a6bf1a03-3c52-4cb2-b6e6-630a29a7afae None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:31:22.958905 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-a6bf1a03-3c52-4cb2-b6e6-630a29a7afae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:31:22.977103 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9251b497-7e38-4857-b132-97ecabbb4dbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:22.977757 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9251b497-7e38-4857-b132-97ecabbb4dbb None admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:22.978044 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9251b497-7e38-4857-b132-97ecabbb4dbb None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:22.978321 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9251b497-7e38-4857-b132-97ecabbb4dbb None admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:22.979009 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9251b497-7e38-4857-b132-97ecabbb4dbb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:22.980461 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9251b497-7e38-4857-b132-97ecabbb4dbb None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:23.007827 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 234/470] 10.222.0.89 () {60 vars in 1132 bytes} [Thu Jul 28 17:31:22 2022] POST /identity/v3/users => generated 248 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:31:23.008640 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9251b497-7e38-4857-b132-97ecabbb4dbb None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:23.039393 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 237/471] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:31:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:31:23.066008 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bf1a03-3c52-4cb2-b6e6-630a29a7afae None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:23.142468 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9251b497-7e38-4857-b132-97ecabbb4dbb None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:23.150694 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:23.153612 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:23.183122 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623ed175-d8dc-4e7f-9d00-6605f4ec4bbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:23.184048 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623ed175-d8dc-4e7f-9d00-6605f4ec4bbb None admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:23.184467 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bf1a03-3c52-4cb2-b6e6-630a29a7afae None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:23.184682 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623ed175-d8dc-4e7f-9d00-6605f4ec4bbb None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:23.185116 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623ed175-d8dc-4e7f-9d00-6605f4ec4bbb None admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:23.186829 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623ed175-d8dc-4e7f-9d00-6605f4ec4bbb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:23.188613 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623ed175-d8dc-4e7f-9d00-6605f4ec4bbb None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:23.212256 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 235/472] 10.222.0.89 () {60 vars in 1132 bytes} [Thu Jul 28 17:31:23 2022] POST /identity/v3/users => generated 248 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:31:23.270236 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a415029-033f-40ab-a7da-7f264ec3fb60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:23.271032 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a415029-033f-40ab-a7da-7f264ec3fb60 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:23.271320 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a415029-033f-40ab-a7da-7f264ec3fb60 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:23.271594 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a415029-033f-40ab-a7da-7f264ec3fb60 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:23.286116 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a415029-033f-40ab-a7da-7f264ec3fb60 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:23.287748 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a415029-033f-40ab-a7da-7f264ec3fb60 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:23.308851 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-2a415029-033f-40ab-a7da-7f264ec3fb60 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:31:23.308851 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:23.308851 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:23.308851 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:23.308851 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:23.308851 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:23.308851 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:23.308851 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:23.308851 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:23.308851 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:23.308851 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:23.308851 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:23.308851 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:31:23.308851 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:31:23.308851 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:31:23.309790 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:31:23.309790 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:23.309790 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:23.309790 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:23.309790 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:23.309790 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:23.309790 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:23.309790 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:23.309790 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:23.309790 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:23.309790 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:23.309790 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:23.309790 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:23.309790 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:23.309790 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:23.309790 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:23.309790 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:23.310621 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:31:23.310621 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:31:23.310621 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:31:23.310621 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:31:23.310621 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 238/473] 10.222.0.89 () {56 vars in 1108 bytes} [Thu Jul 28 17:31:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:31:23.317447 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623ed175-d8dc-4e7f-9d00-6605f4ec4bbb None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:23.382855 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeb4a05-6498-43ae-97ee-1bde849b7334 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:23.383468 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeb4a05-6498-43ae-97ee-1bde849b7334 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:23.383933 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeb4a05-6498-43ae-97ee-1bde849b7334 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:23.385487 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 239/474] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:31:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 17:31:23.393756 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bece3c-32e5-4ebd-8fff-db6f7e7561b1 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:23.394018 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bece3c-32e5-4ebd-8fff-db6f7e7561b1 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:23.394208 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bece3c-32e5-4ebd-8fff-db6f7e7561b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:23.416729 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-40bece3c-32e5-4ebd-8fff-db6f7e7561b1 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:31:23.418974 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-40bece3c-32e5-4ebd-8fff-db6f7e7561b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:31:23.469102 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623ed175-d8dc-4e7f-9d00-6605f4ec4bbb None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:23.521714 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 240/475] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:31:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 17:31:23.524024 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f33d51-d08b-407b-881d-035e1235b382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:23.524672 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f33d51-d08b-407b-881d-035e1235b382 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:23.524947 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f33d51-d08b-407b-881d-035e1235b382 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:23.525246 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f33d51-d08b-407b-881d-035e1235b382 None admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:23.525961 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f33d51-d08b-407b-881d-035e1235b382 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:23.526267 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f33d51-d08b-407b-881d-035e1235b382 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:23.527639 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f33d51-d08b-407b-881d-035e1235b382 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:23.540726 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 236/476] 10.222.0.89 () {56 vars in 1109 bytes} [Thu Jul 28 17:31:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:23.546142 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bece3c-32e5-4ebd-8fff-db6f7e7561b1 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:23.552874 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f33d51-d08b-407b-881d-035e1235b382 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:23.583296 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:23.624795 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:23.676082 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bece3c-32e5-4ebd-8fff-db6f7e7561b1 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:23.740404 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa0e99c-e658-4dd2-b59c-ffe1e68256d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:23.741034 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa0e99c-e658-4dd2-b59c-ffe1e68256d2 None admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:23.741344 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa0e99c-e658-4dd2-b59c-ffe1e68256d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:23.741620 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa0e99c-e658-4dd2-b59c-ffe1e68256d2 None admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:23.742325 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa0e99c-e658-4dd2-b59c-ffe1e68256d2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:23.749299 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa0e99c-e658-4dd2-b59c-ffe1e68256d2 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:23.753767 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f33d51-d08b-407b-881d-035e1235b382 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:23.794326 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5363e16b-a9e6-4209-9bc7-55119d055e3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:23.795569 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5363e16b-a9e6-4209-9bc7-55119d055e3f None admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:23.795866 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5363e16b-a9e6-4209-9bc7-55119d055e3f None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:23.796143 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5363e16b-a9e6-4209-9bc7-55119d055e3f None admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:23.796863 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5363e16b-a9e6-4209-9bc7-55119d055e3f None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:23.797631 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 241/477] 10.222.0.89 () {60 vars in 1132 bytes} [Thu Jul 28 17:31:23 2022] POST /identity/v3/users => generated 249 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:31:23.798352 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5363e16b-a9e6-4209-9bc7-55119d055e3f None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:23.821187 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.sql.core [None req-5363e16b-a9e6-4209-9bc7-55119d055e3f None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 28 17:31:23.821187 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 28 17:31:23.821187 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [parameters: {'id': 'e5c88870c83646f4ad406a5ddbbb89da', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 28 17:31:23.821187 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80325) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 28 17:31:23.843781 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-5363e16b-a9e6-4209-9bc7-55119d055e3f None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 28 17:31:23.843781 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:23.843781 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 28 17:31:23.843781 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 28 17:31:23.843781 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 28 17:31:23.843781 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 28 17:31:23.843781 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 28 17:31:23.843781 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application result = self._query(query) Jul 28 17:31:23.843781 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 28 17:31:23.843781 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application conn.query(q) Jul 28 17:31:23.843781 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 28 17:31:23.843781 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 28 17:31:23.843781 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 28 17:31:23.843781 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application result.read() Jul 28 17:31:23.843781 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 28 17:31:23.844748 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 28 17:31:23.844748 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 28 17:31:23.844748 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 28 17:31:23.844748 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 28 17:31:23.844748 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 28 17:31:23.844748 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 28 17:31:23.844748 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 28 17:31:23.844748 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 28 17:31:23.844748 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:31:23.844748 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 28 17:31:23.844748 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:31:23.844748 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:23.844748 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 28 17:31:23.844748 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 28 17:31:23.844748 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 28 17:31:23.844748 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return ref.to_dict() Jul 28 17:31:23.844748 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 28 17:31:23.844748 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application next(self.gen) Jul 28 17:31:23.846162 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 28 17:31:23.846162 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application yield resource Jul 28 17:31:23.846162 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 28 17:31:23.846162 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application next(self.gen) Jul 28 17:31:23.846162 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 28 17:31:23.846162 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application self.session.rollback() Jul 28 17:31:23.846162 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 28 17:31:23.846162 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application self.force_reraise() Jul 28 17:31:23.846162 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 28 17:31:23.846162 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise self.value Jul 28 17:31:23.846162 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 28 17:31:23.846162 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 28 17:31:23.846162 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 28 17:31:23.846162 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application session.commit() Jul 28 17:31:23.846162 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 28 17:31:23.846162 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 28 17:31:23.846162 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 28 17:31:23.847317 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application self._prepare_impl() Jul 28 17:31:23.847317 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 28 17:31:23.847317 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application self.session.flush() Jul 28 17:31:23.847317 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 28 17:31:23.847317 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application self._flush(objects) Jul 28 17:31:23.847317 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 28 17:31:23.847317 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 28 17:31:23.847317 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 28 17:31:23.847317 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application compat.raise_( Jul 28 17:31:23.847317 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 28 17:31:23.847317 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception Jul 28 17:31:23.847317 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 28 17:31:23.847317 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application flush_context.execute() Jul 28 17:31:23.847317 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 28 17:31:23.847317 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rec.execute(self) Jul 28 17:31:23.847317 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 28 17:31:23.847317 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 28 17:31:23.848192 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 28 17:31:23.848192 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 28 17:31:23.848192 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 28 17:31:23.848192 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 28 17:31:23.848192 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 28 17:31:23.848192 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 28 17:31:23.848192 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 28 17:31:23.848192 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 28 17:31:23.848192 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 28 17:31:23.848192 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 28 17:31:23.848192 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 28 17:31:23.848192 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 28 17:31:23.848192 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 28 17:31:23.848192 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 28 17:31:23.848192 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 28 17:31:23.848192 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception Jul 28 17:31:23.849025 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 28 17:31:23.849025 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 28 17:31:23.849025 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 28 17:31:23.849025 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 28 17:31:23.849025 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 28 17:31:23.849025 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application result = self._query(query) Jul 28 17:31:23.849025 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 28 17:31:23.849025 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application conn.query(q) Jul 28 17:31:23.849025 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 28 17:31:23.849025 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 28 17:31:23.849025 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 28 17:31:23.849025 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application result.read() Jul 28 17:31:23.849025 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 28 17:31:23.849025 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 28 17:31:23.849025 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 28 17:31:23.849025 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 28 17:31:23.854764 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 28 17:31:23.854764 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 28 17:31:23.854764 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 28 17:31:23.854764 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 28 17:31:23.854764 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 28 17:31:23.854764 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 28 17:31:23.854764 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [parameters: {'id': 'e5c88870c83646f4ad406a5ddbbb89da', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 28 17:31:23.854764 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 28 17:31:23.854764 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:31:23.854764 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 17:31:23.854764 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:31:23.854764 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:23.854764 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:23.854764 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:23.854764 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:23.854764 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:23.856782 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:23.856782 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:23.856782 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:23.856782 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:23.856782 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:23.856782 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:23.856782 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 28 17:31:23.856782 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 28 17:31:23.856782 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:23.856782 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:23.856782 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 28 17:31:23.856782 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 28 17:31:23.856782 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 28 17:31:23.856782 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 28 17:31:23.856782 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 28 17:31:23.856782 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:31:23.858911 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 237/478] 10.222.0.89 () {60 vars in 1131 bytes} [Thu Jul 28 17:31:23 2022] POST /identity/v3/roles => generated 144 bytes in 293 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:31:23.859050 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa0e99c-e658-4dd2-b59c-ffe1e68256d2 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:24.016977 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa0e99c-e658-4dd2-b59c-ffe1e68256d2 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:24.029090 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:24.057845 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587e5e83-6709-424e-9107-4373ab2099f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:24.058653 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587e5e83-6709-424e-9107-4373ab2099f1 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:24.059047 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587e5e83-6709-424e-9107-4373ab2099f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:24.059423 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587e5e83-6709-424e-9107-4373ab2099f1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:24.073592 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587e5e83-6709-424e-9107-4373ab2099f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:31:24.079364 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-587e5e83-6709-424e-9107-4373ab2099f1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 17:31:24.079364 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:24.079364 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:24.079364 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:24.079364 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:24.079364 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:24.079364 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:24.079364 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:24.079364 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:24.079364 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:24.079364 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:24.079364 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:24.079364 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:31:24.079364 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:31:24.079364 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:31:24.079364 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:31:24.080457 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:31:24.080457 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:31:24.080457 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:31:24.080457 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:31:24.080457 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:24.080457 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:24.080457 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:24.080457 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:24.080457 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:24.080457 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:24.080457 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:24.080457 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:24.080457 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:24.080457 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:24.080457 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:24.080457 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:24.081545 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:24.081545 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:24.081545 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:24.081545 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:24.081545 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:31:24.081545 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:31:24.081545 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:31:24.081545 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:31:24.081545 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:31:24.081545 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:31:24.081545 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 17:31:24.081545 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:31:24.083574 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 242/479] 10.222.0.89 () {56 vars in 1114 bytes} [Thu Jul 28 17:31:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:31:24.088476 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5363e16b-a9e6-4209-9bc7-55119d055e3f None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:24.189434 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:24.209209 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5363e16b-a9e6-4209-9bc7-55119d055e3f None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:24.250889 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1379452c-80af-428d-8f6c-055add843d15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:24.251523 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1379452c-80af-428d-8f6c-055add843d15 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:24.251807 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1379452c-80af-428d-8f6c-055add843d15 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:24.252081 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1379452c-80af-428d-8f6c-055add843d15 None admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:24.252812 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1379452c-80af-428d-8f6c-055add843d15 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:24.253116 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1379452c-80af-428d-8f6c-055add843d15 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:24.254507 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1379452c-80af-428d-8f6c-055add843d15 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:24.263999 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 238/480] 10.222.0.89 () {56 vars in 1111 bytes} [Thu Jul 28 17:31:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:25.237541 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2be4d32-a439-4052-87c4-219003bd81f7 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:25.238063 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2be4d32-a439-4052-87c4-219003bd81f7 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:25.238063 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2be4d32-a439-4052-87c4-219003bd81f7 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:25.239700 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 243/481] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:31:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:31:25.245403 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19b56d5-199a-4f0c-8069-8018ff2e3178 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:25.245849 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19b56d5-199a-4f0c-8069-8018ff2e3178 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:25.246024 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19b56d5-199a-4f0c-8069-8018ff2e3178 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:25.273835 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-f19b56d5-199a-4f0c-8069-8018ff2e3178 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:31:25.275103 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-f19b56d5-199a-4f0c-8069-8018ff2e3178 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:31:25.375981 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 239/482] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:31:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:31:25.420748 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2be4d32-a439-4052-87c4-219003bd81f7 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:25.631469 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2be4d32-a439-4052-87c4-219003bd81f7 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:25.703077 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f831e97a-2063-449c-aa3c-e82acb895e39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:25.704094 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f831e97a-2063-449c-aa3c-e82acb895e39 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:25.704579 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f831e97a-2063-449c-aa3c-e82acb895e39 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:25.705077 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f831e97a-2063-449c-aa3c-e82acb895e39 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:25.730222 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f831e97a-2063-449c-aa3c-e82acb895e39 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:31:25.736746 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-f831e97a-2063-449c-aa3c-e82acb895e39 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 17:31:25.736746 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:25.736746 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:25.736746 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:25.736746 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:25.736746 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:25.736746 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:25.736746 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:25.736746 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:25.736746 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:25.736746 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:25.736746 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:25.736746 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:31:25.736746 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:31:25.736746 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:31:25.736746 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:31:25.737797 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:31:25.737797 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:31:25.737797 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:31:25.737797 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:31:25.737797 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:25.737797 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:25.737797 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:25.737797 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:25.737797 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:25.737797 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:25.737797 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:25.737797 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:25.737797 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:25.737797 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:25.737797 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:25.737797 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:25.738599 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:25.738599 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:25.738599 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:25.738599 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:25.738599 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:31:25.738599 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:31:25.738599 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:31:25.738599 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:31:25.738599 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:31:25.738599 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:31:25.738599 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 17:31:25.738599 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:31:25.745508 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 244/483] 10.222.0.89 () {56 vars in 1102 bytes} [Thu Jul 28 17:31:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:31:25.751676 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19b56d5-199a-4f0c-8069-8018ff2e3178 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:25.876293 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:25.902988 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19b56d5-199a-4f0c-8069-8018ff2e3178 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:25.910695 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dafdd99-ef0e-468b-8e77-9c5acb6637ed None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:25.910975 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dafdd99-ef0e-468b-8e77-9c5acb6637ed None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:25.911245 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dafdd99-ef0e-468b-8e77-9c5acb6637ed None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:25.912826 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 245/484] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:31:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:31:25.926595 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c713f2b-4861-446a-ba81-7e1c295f8cda None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:25.926879 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c713f2b-4861-446a-ba81-7e1c295f8cda None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:25.927155 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c713f2b-4861-446a-ba81-7e1c295f8cda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:25.955483 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-2c713f2b-4861-446a-ba81-7e1c295f8cda None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:31:25.957287 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-2c713f2b-4861-446a-ba81-7e1c295f8cda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:31:25.959889 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8711996-9384-4c08-bcb9-d1815c216603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:25.960523 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8711996-9384-4c08-bcb9-d1815c216603 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:25.960797 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8711996-9384-4c08-bcb9-d1815c216603 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:25.961067 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8711996-9384-4c08-bcb9-d1815c216603 None admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:25.961829 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8711996-9384-4c08-bcb9-d1815c216603 None admin] RBAC: Adding query filter params (name=service) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:25.962133 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8711996-9384-4c08-bcb9-d1815c216603 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:25.963490 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8711996-9384-4c08-bcb9-d1815c216603 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:25.984969 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 240/485] 10.222.0.89 () {56 vars in 1103 bytes} [Thu Jul 28 17:31:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:25.987152 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8711996-9384-4c08-bcb9-d1815c216603 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:26.011947 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:26.065045 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 246/486] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:31:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:31:26.066332 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701d6a02-676a-44eb-9a13-7dce1a4329b2 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:26.066618 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701d6a02-676a-44eb-9a13-7dce1a4329b2 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:26.066889 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701d6a02-676a-44eb-9a13-7dce1a4329b2 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:26.068544 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 247/487] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:31:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:31:26.073237 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7116982f-cc62-41cc-ac5f-bbd5afbf2af6 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:26.076005 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7116982f-cc62-41cc-ac5f-bbd5afbf2af6 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:26.076396 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7116982f-cc62-41cc-ac5f-bbd5afbf2af6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:26.108254 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-7116982f-cc62-41cc-ac5f-bbd5afbf2af6 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:31:26.110282 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-7116982f-cc62-41cc-ac5f-bbd5afbf2af6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:31:26.147204 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8711996-9384-4c08-bcb9-d1815c216603 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:26.185150 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2139b60d-7da1-4433-a762-dbab367844c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:26.186409 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2139b60d-7da1-4433-a762-dbab367844c2 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:26.186691 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2139b60d-7da1-4433-a762-dbab367844c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:26.186963 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2139b60d-7da1-4433-a762-dbab367844c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:26.196822 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:26.198943 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2139b60d-7da1-4433-a762-dbab367844c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:26.200703 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2139b60d-7da1-4433-a762-dbab367844c2 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:26.207797 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 248/488] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:31:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:31:26.209360 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7116982f-cc62-41cc-ac5f-bbd5afbf2af6 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:26.230027 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-2139b60d-7da1-4433-a762-dbab367844c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:31:26.230027 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:26.230027 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:26.230027 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:26.230027 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:26.230027 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:26.230027 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:26.230027 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:26.230027 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:26.230027 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:26.230027 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:26.230027 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:26.230027 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:31:26.230027 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:31:26.230027 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:31:26.230939 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:31:26.230939 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:26.230939 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:26.230939 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:26.230939 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:26.230939 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:26.230939 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:26.230939 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:26.230939 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:26.230939 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:26.230939 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:26.230939 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:26.230939 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:26.230939 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:26.230939 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:26.230939 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:26.230939 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:26.231781 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:31:26.231781 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:31:26.231781 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:31:26.231781 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:31:26.231781 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 241/489] 10.222.0.89 () {56 vars in 1108 bytes} [Thu Jul 28 17:31:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:31:26.236606 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2139b60d-7da1-4433-a762-dbab367844c2 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:26.328101 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:26.338686 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7116982f-cc62-41cc-ac5f-bbd5afbf2af6 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:26.356279 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2139b60d-7da1-4433-a762-dbab367844c2 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:26.379713 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2acd17-9bcf-42ae-b4e7-6e4779d40562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:26.380359 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2acd17-9bcf-42ae-b4e7-6e4779d40562 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:26.380640 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2acd17-9bcf-42ae-b4e7-6e4779d40562 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:26.380910 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2acd17-9bcf-42ae-b4e7-6e4779d40562 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:26.392657 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2acd17-9bcf-42ae-b4e7-6e4779d40562 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:31:26.397572 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-6d2acd17-9bcf-42ae-b4e7-6e4779d40562 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 17:31:26.397572 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:26.397572 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:26.397572 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:26.397572 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:26.397572 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:26.397572 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:26.397572 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:26.397572 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:26.397572 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:26.397572 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:26.397572 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:26.397572 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:31:26.397572 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:31:26.397572 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:31:26.397572 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:31:26.398478 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:31:26.398478 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:31:26.398478 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:31:26.398478 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:31:26.398478 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:26.398478 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:26.398478 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:26.398478 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:26.398478 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:26.398478 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:26.398478 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:26.398478 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:26.398478 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:26.398478 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:26.398478 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:26.398478 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:26.399266 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:26.399266 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:26.399266 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:26.399266 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:26.399266 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:31:26.399266 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:31:26.399266 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:31:26.399266 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:31:26.399266 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:31:26.399266 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:31:26.399266 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 17:31:26.399266 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:31:26.399266 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 249/490] 10.222.0.89 () {56 vars in 1102 bytes} [Thu Jul 28 17:31:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:31:26.405055 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2acd17-9bcf-42ae-b4e7-6e4779d40562 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:26.407935 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1e1e35-aef4-4e97-bfb5-c0e1e91cc882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:26.409101 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1e1e35-aef4-4e97-bfb5-c0e1e91cc882 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:26.410644 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1e1e35-aef4-4e97-bfb5-c0e1e91cc882 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:26.411122 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1e1e35-aef4-4e97-bfb5-c0e1e91cc882 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:26.426137 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1e1e35-aef4-4e97-bfb5-c0e1e91cc882 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:31:26.435433 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-3a1e1e35-aef4-4e97-bfb5-c0e1e91cc882 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 17:31:26.435433 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:26.435433 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:26.435433 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:26.435433 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:26.435433 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:26.435433 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:26.435433 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:26.435433 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:26.435433 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:26.435433 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:26.435433 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:26.435433 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:31:26.435433 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:31:26.435433 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:31:26.435433 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:31:26.437334 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:31:26.437334 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:31:26.437334 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:31:26.437334 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:31:26.437334 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:26.437334 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:26.437334 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:26.437334 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:26.437334 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:26.437334 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:26.437334 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:26.437334 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:26.437334 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:26.437334 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:26.437334 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:26.437334 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:26.438161 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:26.438161 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:26.438161 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:26.438161 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:26.438161 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:31:26.438161 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:31:26.438161 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:31:26.438161 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:31:26.438161 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:31:26.438161 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:31:26.438161 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 17:31:26.438161 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:31:26.438870 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 242/491] 10.222.0.89 () {56 vars in 1102 bytes} [Thu Jul 28 17:31:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:31:26.439850 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1e1e35-aef4-4e97-bfb5-c0e1e91cc882 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:26.504897 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:26.555299 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2acd17-9bcf-42ae-b4e7-6e4779d40562 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:26.557054 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1e1e35-aef4-4e97-bfb5-c0e1e91cc882 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:26.594040 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f8a816-ea4d-415f-ba10-8944e9f77e47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:26.594709 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f8a816-ea4d-415f-ba10-8944e9f77e47 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:26.594991 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f8a816-ea4d-415f-ba10-8944e9f77e47 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:26.595260 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f8a816-ea4d-415f-ba10-8944e9f77e47 None admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:26.595976 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f8a816-ea4d-415f-ba10-8944e9f77e47 None admin] RBAC: Adding query filter params (name=service) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:26.596404 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f8a816-ea4d-415f-ba10-8944e9f77e47 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:26.597748 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f8a816-ea4d-415f-ba10-8944e9f77e47 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:26.603642 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb268d4-7ec3-4831-9c90-2fe16e529f50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:26.604873 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb268d4-7ec3-4831-9c90-2fe16e529f50 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:26.605390 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb268d4-7ec3-4831-9c90-2fe16e529f50 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:26.605887 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb268d4-7ec3-4831-9c90-2fe16e529f50 None admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:26.606804 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb268d4-7ec3-4831-9c90-2fe16e529f50 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:26.607309 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb268d4-7ec3-4831-9c90-2fe16e529f50 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:26.609697 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb268d4-7ec3-4831-9c90-2fe16e529f50 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:26.611828 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 243/492] 10.222.0.89 () {56 vars in 1103 bytes} [Thu Jul 28 17:31:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:26.613183 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f8a816-ea4d-415f-ba10-8944e9f77e47 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:26.626839 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 250/493] 10.222.0.89 () {56 vars in 1109 bytes} [Thu Jul 28 17:31:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:26.628239 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb268d4-7ec3-4831-9c90-2fe16e529f50 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:26.645296 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:26.730493 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f8a816-ea4d-415f-ba10-8944e9f77e47 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:26.739900 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb268d4-7ec3-4831-9c90-2fe16e529f50 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:26.746920 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:26.770737 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1986665f-b3ec-484c-83f8-a505cba426ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:26.771370 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1986665f-b3ec-484c-83f8-a505cba426ed None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:26.771642 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1986665f-b3ec-484c-83f8-a505cba426ed None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:26.771914 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1986665f-b3ec-484c-83f8-a505cba426ed None admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:26.772632 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1986665f-b3ec-484c-83f8-a505cba426ed None admin] RBAC: Adding query filter params (name=service) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:26.772938 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1986665f-b3ec-484c-83f8-a505cba426ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:26.774359 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1986665f-b3ec-484c-83f8-a505cba426ed None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:26.775214 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76499049-119d-45b9-a1ec-0f3829b25dc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:26.775853 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76499049-119d-45b9-a1ec-0f3829b25dc8 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:26.776132 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76499049-119d-45b9-a1ec-0f3829b25dc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:26.776396 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76499049-119d-45b9-a1ec-0f3829b25dc8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:26.788299 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 244/494] 10.222.0.89 () {56 vars in 1103 bytes} [Thu Jul 28 17:31:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:26.788299 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1986665f-b3ec-484c-83f8-a505cba426ed None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:26.793959 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76499049-119d-45b9-a1ec-0f3829b25dc8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:26.795556 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76499049-119d-45b9-a1ec-0f3829b25dc8 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:26.811636 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-76499049-119d-45b9-a1ec-0f3829b25dc8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:31:26.811636 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:26.811636 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:26.811636 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:26.811636 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:26.811636 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:26.811636 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:26.811636 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:26.811636 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:26.811636 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:26.811636 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:26.811636 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:26.811636 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:31:26.811636 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:31:26.811636 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:31:26.812558 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:31:26.812558 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:26.812558 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:26.812558 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:26.812558 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:26.812558 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:26.812558 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:26.812558 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:26.812558 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:26.812558 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:26.812558 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:26.812558 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:26.812558 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:26.812558 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:26.812558 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:26.812558 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:26.812558 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:26.813418 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:31:26.813418 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:31:26.813418 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:31:26.813418 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:31:26.817083 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 251/495] 10.222.0.89 () {56 vars in 1108 bytes} [Thu Jul 28 17:31:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:31:26.818314 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892471aa-5103-4c90-b9d2-fbca31b4b0d9 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:26.818581 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892471aa-5103-4c90-b9d2-fbca31b4b0d9 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:26.818865 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892471aa-5103-4c90-b9d2-fbca31b4b0d9 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:26.821185 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 252/496] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:31:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:31:26.821185 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892471aa-5103-4c90-b9d2-fbca31b4b0d9 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:26.897068 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:26.908551 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1986665f-b3ec-484c-83f8-a505cba426ed None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:26.945291 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d9da1b-b33a-4083-9aeb-43bc3145b6a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:26.945959 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d9da1b-b33a-4083-9aeb-43bc3145b6a7 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:26.946238 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d9da1b-b33a-4083-9aeb-43bc3145b6a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:26.946337 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892471aa-5103-4c90-b9d2-fbca31b4b0d9 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:26.946508 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d9da1b-b33a-4083-9aeb-43bc3145b6a7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:26.956570 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d9da1b-b33a-4083-9aeb-43bc3145b6a7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:26.958390 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d9da1b-b33a-4083-9aeb-43bc3145b6a7 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:26.972396 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-93d9da1b-b33a-4083-9aeb-43bc3145b6a7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 17:31:26.972396 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:26.972396 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:26.972396 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:26.972396 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:26.972396 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:26.972396 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:26.972396 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:26.972396 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:26.972396 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:26.972396 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:26.972396 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:26.972396 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 17:31:26.972396 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 17:31:26.972396 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 17:31:26.972396 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 17:31:26.973429 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:26.973429 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:26.973429 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 17:31:26.973429 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:31:26.973429 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 17:31:26.973429 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:31:26.973429 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:26.973429 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:26.973429 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:26.973429 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:26.973429 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:26.973429 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:26.973429 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:26.973429 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:26.973429 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:26.973429 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:26.973429 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:26.973429 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:26.974393 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:26.974393 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:26.974393 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 17:31:26.974393 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 17:31:26.974393 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 17:31:26.974393 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 17:31:26.974393 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 17:31:26.974393 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 17:31:26.974393 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 17:31:26.974393 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:31:26.974393 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 245/497] 10.222.0.89 () {56 vars in 1093 bytes} [Thu Jul 28 17:31:26 2022] GET /identity/v3/users/nova => generated 82 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:31:26.974952 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d9da1b-b33a-4083-9aeb-43bc3145b6a7 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:26.989489 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219c353f-00be-4412-a5f3-6d14fccf167b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:26.990097 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219c353f-00be-4412-a5f3-6d14fccf167b None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:26.990331 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219c353f-00be-4412-a5f3-6d14fccf167b None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:26.990592 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219c353f-00be-4412-a5f3-6d14fccf167b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:27.003506 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219c353f-00be-4412-a5f3-6d14fccf167b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:27.005493 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219c353f-00be-4412-a5f3-6d14fccf167b None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:27.016506 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:27.019970 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-219c353f-00be-4412-a5f3-6d14fccf167b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:31:27.019970 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:27.019970 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:27.019970 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:27.019970 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:27.019970 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:27.019970 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:27.019970 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:27.019970 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:27.019970 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:27.019970 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:27.019970 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:27.019970 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:31:27.019970 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:31:27.019970 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:31:27.020803 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:31:27.020803 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:27.020803 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:27.020803 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:27.020803 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:27.020803 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:27.020803 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:27.020803 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:27.020803 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:27.020803 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:27.020803 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:27.020803 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:27.020803 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:27.020803 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:27.020803 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:27.020803 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:27.020803 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:27.022250 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:31:27.022250 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:31:27.022250 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:31:27.022250 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:31:27.022250 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 253/498] 10.222.0.89 () {56 vars in 1108 bytes} [Thu Jul 28 17:31:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:31:27.022645 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cebd2d-979e-43dc-9369-e3169955ea49 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:27.022900 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cebd2d-979e-43dc-9369-e3169955ea49 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:27.023170 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cebd2d-979e-43dc-9369-e3169955ea49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:27.046523 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-d1cebd2d-979e-43dc-9369-e3169955ea49 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:31:27.048039 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-d1cebd2d-979e-43dc-9369-e3169955ea49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:31:27.087628 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d9da1b-b33a-4083-9aeb-43bc3145b6a7 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:27.118792 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 254/499] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:31:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:31:27.119901 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cebd2d-979e-43dc-9369-e3169955ea49 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:27.133230 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9c5119-3757-4354-a603-fa35a077213e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:27.133859 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9c5119-3757-4354-a603-fa35a077213e None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:27.134144 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9c5119-3757-4354-a603-fa35a077213e None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:27.134420 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9c5119-3757-4354-a603-fa35a077213e None admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:27.135119 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9c5119-3757-4354-a603-fa35a077213e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:27.135425 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9c5119-3757-4354-a603-fa35a077213e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:27.136815 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9c5119-3757-4354-a603-fa35a077213e None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:27.148924 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 246/500] 10.222.0.89 () {56 vars in 1109 bytes} [Thu Jul 28 17:31:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:27.149791 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9c5119-3757-4354-a603-fa35a077213e None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:27.167588 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:27.220949 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cebd2d-979e-43dc-9369-e3169955ea49 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:27.242554 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:27.249092 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9c5119-3757-4354-a603-fa35a077213e None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:27.255034 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7089254-eb09-4535-ad71-ebf4ab0d7acc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:27.255647 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7089254-eb09-4535-ad71-ebf4ab0d7acc None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:27.255928 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7089254-eb09-4535-ad71-ebf4ab0d7acc None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:27.256198 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7089254-eb09-4535-ad71-ebf4ab0d7acc None admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:27.256989 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7089254-eb09-4535-ad71-ebf4ab0d7acc None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:27.257322 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7089254-eb09-4535-ad71-ebf4ab0d7acc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:27.258748 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7089254-eb09-4535-ad71-ebf4ab0d7acc None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:27.284144 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d790dea9-2f11-438e-bd5a-d100a8dd0a1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:27.284748 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d790dea9-2f11-438e-bd5a-d100a8dd0a1e None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:27.285022 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d790dea9-2f11-438e-bd5a-d100a8dd0a1e None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:27.285323 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d790dea9-2f11-438e-bd5a-d100a8dd0a1e None admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:27.286004 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d790dea9-2f11-438e-bd5a-d100a8dd0a1e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:27.286303 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d790dea9-2f11-438e-bd5a-d100a8dd0a1e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:27.287656 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d790dea9-2f11-438e-bd5a-d100a8dd0a1e None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:27.294589 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 255/501] 10.222.0.89 () {56 vars in 1133 bytes} [Thu Jul 28 17:31:27 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:27.296369 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d5b661-5555-4ea7-a329-22383d08d469 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:27.296657 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d5b661-5555-4ea7-a329-22383d08d469 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:27.296928 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d5b661-5555-4ea7-a329-22383d08d469 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:27.298326 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 256/502] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:31:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:31:27.298854 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d5b661-5555-4ea7-a329-22383d08d469 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:27.304297 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 247/503] 10.222.0.89 () {56 vars in 1109 bytes} [Thu Jul 28 17:31:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:27.305118 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d790dea9-2f11-438e-bd5a-d100a8dd0a1e None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:27.402406 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d5b661-5555-4ea7-a329-22383d08d469 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:27.402967 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d790dea9-2f11-438e-bd5a-d100a8dd0a1e None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:27.403913 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:27.437528 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039bc785-2549-43b1-a385-eda89f9cc8e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:27.438153 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039bc785-2549-43b1-a385-eda89f9cc8e3 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:27.438430 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039bc785-2549-43b1-a385-eda89f9cc8e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:27.438658 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08774a8-8fd0-4bc8-837b-72d650319165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:27.438779 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039bc785-2549-43b1-a385-eda89f9cc8e3 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:27.439290 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08774a8-8fd0-4bc8-837b-72d650319165 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:27.439572 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08774a8-8fd0-4bc8-837b-72d650319165 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:27.439846 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08774a8-8fd0-4bc8-837b-72d650319165 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:27.446125 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039bc785-2549-43b1-a385-eda89f9cc8e3 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:27.447562 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039bc785-2549-43b1-a385-eda89f9cc8e3 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:27.450079 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08774a8-8fd0-4bc8-837b-72d650319165 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:31:27.455585 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-a08774a8-8fd0-4bc8-837b-72d650319165 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 17:31:27.455585 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:27.455585 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:27.455585 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:27.455585 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:27.455585 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:27.455585 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:27.455585 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:27.455585 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:27.455585 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:27.455585 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:27.455585 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:27.455585 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:31:27.455585 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:31:27.455585 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:31:27.455585 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:31:27.456474 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:31:27.456474 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:31:27.456474 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:31:27.456474 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:31:27.456474 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:27.456474 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:27.456474 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:27.456474 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:27.456474 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:27.456474 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:27.456474 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:27.456474 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:27.456474 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:27.456474 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:27.456474 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:27.456474 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:27.457296 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:27.457296 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:27.457296 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:27.457296 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:27.457296 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:31:27.457296 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:31:27.457296 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:31:27.457296 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:31:27.457296 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:31:27.457296 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:31:27.457296 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 17:31:27.457296 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:31:27.457296 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 257/504] 10.222.0.89 () {56 vars in 1102 bytes} [Thu Jul 28 17:31:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:31:27.457976 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08774a8-8fd0-4bc8-837b-72d650319165 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:27.458923 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-039bc785-2549-43b1-a385-eda89f9cc8e3 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 28 17:31:27.458923 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:27.458923 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:27.458923 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:27.458923 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:27.458923 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:27.458923 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:27.458923 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:27.458923 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:27.458923 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:27.458923 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:27.458923 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:27.458923 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 17:31:27.458923 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 17:31:27.458923 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 17:31:27.458923 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 17:31:27.459910 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:27.459910 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:27.459910 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 17:31:27.459910 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:31:27.459910 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 17:31:27.459910 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:31:27.459910 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:27.459910 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:27.459910 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:27.459910 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:27.459910 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:27.459910 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:27.459910 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:27.459910 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:27.459910 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:27.459910 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:27.459910 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:27.459910 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:27.460782 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:27.460782 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:27.460782 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 17:31:27.460782 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 17:31:27.460782 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 17:31:27.460782 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 17:31:27.460782 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 17:31:27.460782 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 17:31:27.460782 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 28 17:31:27.460782 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:31:27.460782 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 248/505] 10.222.0.89 () {56 vars in 1099 bytes} [Thu Jul 28 17:31:27 2022] GET /identity/v3/users/cinder => generated 84 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:31:27.462155 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9724a646-596c-40ef-8ec5-de24affb0971 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:27.462155 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9724a646-596c-40ef-8ec5-de24affb0971 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:27.462531 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9724a646-596c-40ef-8ec5-de24affb0971 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:27.480202 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-9724a646-596c-40ef-8ec5-de24affb0971 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:31:27.481580 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-9724a646-596c-40ef-8ec5-de24affb0971 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:31:27.493791 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:27.550965 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 249/506] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:31:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:31:27.551813 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9724a646-596c-40ef-8ec5-de24affb0971 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:27.579189 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08774a8-8fd0-4bc8-837b-72d650319165 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:27.614538 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532b1cd5-d1ee-4804-98d1-2f8d7391817f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:27.615142 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532b1cd5-d1ee-4804-98d1-2f8d7391817f None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:27.615419 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532b1cd5-d1ee-4804-98d1-2f8d7391817f None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:27.615689 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532b1cd5-d1ee-4804-98d1-2f8d7391817f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:27.645913 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:27.650839 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9724a646-596c-40ef-8ec5-de24affb0971 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:27.684966 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532b1cd5-d1ee-4804-98d1-2f8d7391817f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:27.685339 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0f7117-a5cb-4366-9545-e338f616139b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:27.686051 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0f7117-a5cb-4366-9545-e338f616139b None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:27.686437 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0f7117-a5cb-4366-9545-e338f616139b None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:27.686531 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532b1cd5-d1ee-4804-98d1-2f8d7391817f None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:27.686713 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0f7117-a5cb-4366-9545-e338f616139b None admin] PATH_INFO: `/v3/users/glance` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:27.694140 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0f7117-a5cb-4366-9545-e338f616139b None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:27.695571 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0f7117-a5cb-4366-9545-e338f616139b None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:27.707925 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-8c0f7117-a5cb-4366-9545-e338f616139b None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 28 17:31:27.707925 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:27.707925 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:27.707925 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:27.707925 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:27.707925 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:27.707925 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:27.707925 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:27.707925 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:27.707925 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:27.707925 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:27.707925 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:27.707925 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 17:31:27.707925 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 17:31:27.707925 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 17:31:27.707925 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 17:31:27.708938 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:27.708938 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:27.708938 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 17:31:27.708938 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:31:27.708938 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 17:31:27.708938 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:31:27.708938 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:27.708938 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:27.708938 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:27.708938 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:27.708938 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:27.708938 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:27.708938 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:27.708938 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:27.708938 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:27.708938 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:27.708938 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:27.708938 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:27.709864 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:27.709864 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:27.709864 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 17:31:27.709864 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 17:31:27.709864 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 17:31:27.709864 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 17:31:27.709864 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 17:31:27.709864 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 17:31:27.709864 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 28 17:31:27.709864 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:31:27.709864 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 250/507] 10.222.0.89 () {56 vars in 1099 bytes} [Thu Jul 28 17:31:27 2022] GET /identity/v3/users/glance => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:31:27.710417 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0f7117-a5cb-4366-9545-e338f616139b None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:27.719408 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:27.791192 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 258/508] 10.222.0.89 () {58 vars in 1282 bytes} [Thu Jul 28 17:31:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 17:31:27.792054 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532b1cd5-d1ee-4804-98d1-2f8d7391817f None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:27.808580 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0f7117-a5cb-4366-9545-e338f616139b None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:27.844222 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c886c326-102c-42e5-b73a-a50043603375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:27.844842 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c886c326-102c-42e5-b73a-a50043603375 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:27.845123 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c886c326-102c-42e5-b73a-a50043603375 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:27.845429 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c886c326-102c-42e5-b73a-a50043603375 None admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:27.846146 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c886c326-102c-42e5-b73a-a50043603375 None admin] RBAC: Adding query filter params (name=service) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:27.846451 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c886c326-102c-42e5-b73a-a50043603375 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:27.847840 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c886c326-102c-42e5-b73a-a50043603375 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:27.855883 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:27.856065 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 251/509] 10.222.0.89 () {56 vars in 1103 bytes} [Thu Jul 28 17:31:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:27.856826 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c886c326-102c-42e5-b73a-a50043603375 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:27.895420 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532b1cd5-d1ee-4804-98d1-2f8d7391817f None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:27.926518 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:27.930345 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d8c904-ed46-490c-9548-6bd05744a180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:27.930953 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d8c904-ed46-490c-9548-6bd05744a180 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:27.931235 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d8c904-ed46-490c-9548-6bd05744a180 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:27.931503 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d8c904-ed46-490c-9548-6bd05744a180 None admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:27.932295 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d8c904-ed46-490c-9548-6bd05744a180 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:27.932610 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d8c904-ed46-490c-9548-6bd05744a180 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:27.934052 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d8c904-ed46-490c-9548-6bd05744a180 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:27.952341 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 259/510] 10.222.0.89 () {56 vars in 1137 bytes} [Thu Jul 28 17:31:27 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:27.956917 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d8c904-ed46-490c-9548-6bd05744a180 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:27.962032 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c886c326-102c-42e5-b73a-a50043603375 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:27.995277 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53587ba8-fa68-45d2-a0ae-88857b82bc3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:27.995921 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53587ba8-fa68-45d2-a0ae-88857b82bc3a None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:27.996202 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53587ba8-fa68-45d2-a0ae-88857b82bc3a None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:27.996479 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53587ba8-fa68-45d2-a0ae-88857b82bc3a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:28.008119 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53587ba8-fa68-45d2-a0ae-88857b82bc3a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:28.009609 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53587ba8-fa68-45d2-a0ae-88857b82bc3a None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:28.024635 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-53587ba8-fa68-45d2-a0ae-88857b82bc3a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:31:28.024635 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:28.024635 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:28.024635 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:28.024635 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:28.024635 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:28.024635 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:28.024635 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:28.024635 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:28.024635 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:28.024635 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:28.024635 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:28.024635 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:31:28.024635 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:31:28.024635 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:31:28.025738 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:31:28.025738 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:28.025738 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:28.025738 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:28.025738 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:28.025738 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:28.025738 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:28.025738 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:28.025738 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:28.025738 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:28.025738 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:28.025738 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:28.025738 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:28.025738 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:28.025738 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:28.025738 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:28.025738 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:28.026652 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:31:28.026652 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:31:28.026652 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:31:28.026652 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:31:28.026652 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 252/511] 10.222.0.89 () {56 vars in 1108 bytes} [Thu Jul 28 17:31:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:31:28.026652 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53587ba8-fa68-45d2-a0ae-88857b82bc3a None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:28.056439 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d8c904-ed46-490c-9548-6bd05744a180 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:28.077856 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:28.092841 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbbb1f8-0446-4b6e-99c0-22fb73170568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:28.093618 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbbb1f8-0446-4b6e-99c0-22fb73170568 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:28.094033 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbbb1f8-0446-4b6e-99c0-22fb73170568 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:28.094393 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbbb1f8-0446-4b6e-99c0-22fb73170568 None admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:28.095266 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbbb1f8-0446-4b6e-99c0-22fb73170568 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:28.095657 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbbb1f8-0446-4b6e-99c0-22fb73170568 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:28.097211 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbbb1f8-0446-4b6e-99c0-22fb73170568 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:28.118190 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 260/512] 10.222.0.89 () {56 vars in 1137 bytes} [Thu Jul 28 17:31:27 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:28.118190 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbbb1f8-0446-4b6e-99c0-22fb73170568 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:28.131909 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53587ba8-fa68-45d2-a0ae-88857b82bc3a None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:28.152245 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:28.166969 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c418ed58-b6ed-4de9-bcd0-d535f28bc083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:28.167611 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c418ed58-b6ed-4de9-bcd0-d535f28bc083 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:28.167893 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c418ed58-b6ed-4de9-bcd0-d535f28bc083 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:28.168167 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c418ed58-b6ed-4de9-bcd0-d535f28bc083 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:28.178887 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c418ed58-b6ed-4de9-bcd0-d535f28bc083 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:28.180333 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c418ed58-b6ed-4de9-bcd0-d535f28bc083 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:28.198111 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-c418ed58-b6ed-4de9-bcd0-d535f28bc083 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:31:28.198111 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:28.198111 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:28.198111 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:28.198111 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:28.198111 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:28.198111 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:28.198111 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:28.198111 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:28.198111 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:28.198111 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:28.198111 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:28.198111 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:31:28.198111 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:31:28.198111 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:31:28.199087 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:31:28.199087 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:28.199087 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:28.199087 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:28.199087 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:28.199087 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:28.199087 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:28.199087 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:28.199087 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:28.199087 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:28.199087 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:28.199087 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:28.199087 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:28.199087 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:28.199087 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:28.199087 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:28.199087 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:28.199941 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:31:28.199941 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:31:28.199941 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:31:28.199941 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:31:28.199941 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 253/513] 10.222.0.89 () {56 vars in 1108 bytes} [Thu Jul 28 17:31:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:31:28.200201 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c418ed58-b6ed-4de9-bcd0-d535f28bc083 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:28.216526 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbbb1f8-0446-4b6e-99c0-22fb73170568 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:28.252280 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3199410-3a61-419d-8ecd-012f55d5de41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:28.252917 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3199410-3a61-419d-8ecd-012f55d5de41 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:28.253226 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3199410-3a61-419d-8ecd-012f55d5de41 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:28.253509 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3199410-3a61-419d-8ecd-012f55d5de41 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:28.263324 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3199410-3a61-419d-8ecd-012f55d5de41 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:28.265620 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3199410-3a61-419d-8ecd-012f55d5de41 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:28.282246 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-d3199410-3a61-419d-8ecd-012f55d5de41 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:31:28.282246 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:28.282246 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:28.282246 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:28.282246 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:28.282246 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:28.282246 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:28.282246 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:28.282246 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:28.282246 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:28.282246 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:28.282246 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:28.282246 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:31:28.282246 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:31:28.282246 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:31:28.283099 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:31:28.283099 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:28.283099 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:28.283099 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:28.283099 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:28.283099 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:28.283099 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:28.283099 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:28.283099 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:28.283099 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:28.283099 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:28.283099 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:28.283099 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:28.283099 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:28.283099 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:28.283099 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:28.283099 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:28.283969 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:31:28.283969 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:31:28.283969 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:31:28.283969 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:31:28.283969 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 261/514] 10.222.0.89 () {56 vars in 1108 bytes} [Thu Jul 28 17:31:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:31:28.284245 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3199410-3a61-419d-8ecd-012f55d5de41 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:28.310220 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c418ed58-b6ed-4de9-bcd0-d535f28bc083 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:28.314141 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:28.344787 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249f51c1-cedf-4352-8c31-f8244f6d7bb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:28.345429 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249f51c1-cedf-4352-8c31-f8244f6d7bb4 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:28.345710 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249f51c1-cedf-4352-8c31-f8244f6d7bb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:28.345980 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249f51c1-cedf-4352-8c31-f8244f6d7bb4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:28.385642 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3199410-3a61-419d-8ecd-012f55d5de41 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:28.387241 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:28.414702 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249f51c1-cedf-4352-8c31-f8244f6d7bb4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:28.416139 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249f51c1-cedf-4352-8c31-f8244f6d7bb4 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:28.420622 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90e2089-c607-4952-95f5-49c1a55b0291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:28.421264 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90e2089-c607-4952-95f5-49c1a55b0291 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:28.421543 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90e2089-c607-4952-95f5-49c1a55b0291 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:28.421816 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90e2089-c607-4952-95f5-49c1a55b0291 None admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:28.422498 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90e2089-c607-4952-95f5-49c1a55b0291 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:28.422797 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90e2089-c607-4952-95f5-49c1a55b0291 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:28.424137 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90e2089-c607-4952-95f5-49c1a55b0291 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:28.435938 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 262/515] 10.222.0.89 () {56 vars in 1109 bytes} [Thu Jul 28 17:31:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:28.436690 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90e2089-c607-4952-95f5-49c1a55b0291 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:28.521704 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 254/516] 10.222.0.89 () {58 vars in 1282 bytes} [Thu Jul 28 17:31:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 17:31:28.522543 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249f51c1-cedf-4352-8c31-f8244f6d7bb4 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:28.526815 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:28.536543 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90e2089-c607-4952-95f5-49c1a55b0291 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:28.570500 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595191a2-0ab6-4246-b852-69c1046dbce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:28.571026 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595191a2-0ab6-4246-b852-69c1046dbce0 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:28.571301 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595191a2-0ab6-4246-b852-69c1046dbce0 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:28.571565 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595191a2-0ab6-4246-b852-69c1046dbce0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:28.594764 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:28.624860 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249f51c1-cedf-4352-8c31-f8244f6d7bb4 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:28.638640 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595191a2-0ab6-4246-b852-69c1046dbce0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:28.640732 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595191a2-0ab6-4246-b852-69c1046dbce0 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:28.660049 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab41eed3-f625-4b06-bb29-4ca77cd787dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:28.660696 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab41eed3-f625-4b06-bb29-4ca77cd787dc None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:28.660981 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab41eed3-f625-4b06-bb29-4ca77cd787dc None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:28.661281 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab41eed3-f625-4b06-bb29-4ca77cd787dc None admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:28.661974 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab41eed3-f625-4b06-bb29-4ca77cd787dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:28.662312 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab41eed3-f625-4b06-bb29-4ca77cd787dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:28.663691 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab41eed3-f625-4b06-bb29-4ca77cd787dc None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:28.674985 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 255/517] 10.222.0.89 () {56 vars in 1109 bytes} [Thu Jul 28 17:31:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:28.675778 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab41eed3-f625-4b06-bb29-4ca77cd787dc None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:28.730445 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:28.742017 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 263/518] 10.222.0.89 () {58 vars in 1282 bytes} [Thu Jul 28 17:31:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 17:31:28.742759 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595191a2-0ab6-4246-b852-69c1046dbce0 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:28.777957 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab41eed3-f625-4b06-bb29-4ca77cd787dc None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:28.807274 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:28.814599 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec08be40-15f3-4dcd-a50f-fe6d8b678a6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:28.815259 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec08be40-15f3-4dcd-a50f-fe6d8b678a6a None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:28.815598 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec08be40-15f3-4dcd-a50f-fe6d8b678a6a None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:28.815875 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec08be40-15f3-4dcd-a50f-fe6d8b678a6a None admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:28.816577 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec08be40-15f3-4dcd-a50f-fe6d8b678a6a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:28.816886 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec08be40-15f3-4dcd-a50f-fe6d8b678a6a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:28.818292 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec08be40-15f3-4dcd-a50f-fe6d8b678a6a None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:28.834035 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 256/519] 10.222.0.89 () {56 vars in 1109 bytes} [Thu Jul 28 17:31:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:28.834870 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec08be40-15f3-4dcd-a50f-fe6d8b678a6a None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:28.844981 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595191a2-0ab6-4246-b852-69c1046dbce0 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:28.879584 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41d3505-9a81-4ff2-82a9-06d0ca26335a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:28.880173 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41d3505-9a81-4ff2-82a9-06d0ca26335a None admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:28.880449 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41d3505-9a81-4ff2-82a9-06d0ca26335a None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:28.880726 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41d3505-9a81-4ff2-82a9-06d0ca26335a None admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:28.881421 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41d3505-9a81-4ff2-82a9-06d0ca26335a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:28.882850 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41d3505-9a81-4ff2-82a9-06d0ca26335a None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:28.903483 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 264/520] 10.222.0.89 () {60 vars in 1132 bytes} [Thu Jul 28 17:31:28 2022] POST /identity/v3/users => generated 247 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:31:28.904498 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41d3505-9a81-4ff2-82a9-06d0ca26335a None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:28.942357 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec08be40-15f3-4dcd-a50f-fe6d8b678a6a None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:28.957125 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:28.986386 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d0c82b-0328-4de7-91c4-263d3f96a48c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:28.987056 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d0c82b-0328-4de7-91c4-263d3f96a48c None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:28.987332 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d0c82b-0328-4de7-91c4-263d3f96a48c None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:28.987601 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d0c82b-0328-4de7-91c4-263d3f96a48c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:29.001463 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d0c82b-0328-4de7-91c4-263d3f96a48c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:29.002932 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d0c82b-0328-4de7-91c4-263d3f96a48c None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:29.018792 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41d3505-9a81-4ff2-82a9-06d0ca26335a None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:29.019730 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-04d0c82b-0328-4de7-91c4-263d3f96a48c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:31:29.019730 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:29.019730 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:29.019730 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:29.019730 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:29.019730 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:29.019730 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:29.019730 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:29.019730 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:29.019730 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:29.019730 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:29.019730 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:29.019730 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:31:29.019730 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:31:29.019730 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:31:29.020550 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:31:29.020550 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:29.020550 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:29.020550 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:29.020550 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:29.020550 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:29.020550 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:29.020550 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:29.020550 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:29.020550 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:29.020550 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:29.020550 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:29.020550 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:29.020550 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:29.020550 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:29.020550 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:29.020550 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:29.021465 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:31:29.021465 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:31:29.021465 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:31:29.021465 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:31:29.021465 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 257/521] 10.222.0.89 () {56 vars in 1108 bytes} [Thu Jul 28 17:31:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:31:29.022272 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d0c82b-0328-4de7-91c4-263d3f96a48c None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:29.065750 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01958d1b-353e-4f8c-9371-094f22efa199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:29.066098 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01958d1b-353e-4f8c-9371-094f22efa199 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:29.066384 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01958d1b-353e-4f8c-9371-094f22efa199 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:29.066659 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01958d1b-353e-4f8c-9371-094f22efa199 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:29.075798 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01958d1b-353e-4f8c-9371-094f22efa199 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:29.077964 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01958d1b-353e-4f8c-9371-094f22efa199 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:29.078075 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:29.090757 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-01958d1b-353e-4f8c-9371-094f22efa199 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:31:29.090757 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:29.090757 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:29.090757 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:29.090757 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:29.090757 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:29.090757 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:29.090757 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:29.090757 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:29.090757 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:29.090757 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:29.090757 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:29.090757 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 17:31:29.090757 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 17:31:29.090757 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 17:31:29.091618 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 17:31:29.091618 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:29.091618 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:29.091618 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:29.091618 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:29.091618 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:29.091618 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:29.091618 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:29.091618 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:29.091618 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:29.091618 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:29.091618 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:29.091618 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:29.091618 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:29.091618 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:29.091618 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:29.091618 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:29.093152 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 17:31:29.093152 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 17:31:29.093152 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 17:31:29.093152 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 17:31:29.093152 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 17:31:29.093152 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 17:31:29.093152 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:31:29.093152 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:31:29.093152 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 265/522] 10.222.0.89 () {56 vars in 1111 bytes} [Thu Jul 28 17:31:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:31:29.094429 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01958d1b-353e-4f8c-9371-094f22efa199 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:29.145587 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d0c82b-0328-4de7-91c4-263d3f96a48c None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:29.180910 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f77e4d5-767d-4910-81a8-f626094fd769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:29.181673 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f77e4d5-767d-4910-81a8-f626094fd769 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:29.181951 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f77e4d5-767d-4910-81a8-f626094fd769 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:29.182221 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f77e4d5-767d-4910-81a8-f626094fd769 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:29.194095 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f77e4d5-767d-4910-81a8-f626094fd769 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:29.195543 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f77e4d5-767d-4910-81a8-f626094fd769 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:29.207725 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01958d1b-353e-4f8c-9371-094f22efa199 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:29.210488 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-6f77e4d5-767d-4910-81a8-f626094fd769 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:31:29.210488 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:29.210488 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:29.210488 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:29.210488 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:29.210488 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:29.210488 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:29.210488 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:29.210488 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:29.210488 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:29.210488 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:29.210488 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:29.210488 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:31:29.210488 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:31:29.210488 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:31:29.211514 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:31:29.211514 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:29.211514 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:29.211514 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:29.211514 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:29.211514 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:29.211514 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:29.211514 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:29.211514 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:29.211514 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:29.211514 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:29.211514 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:29.211514 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:29.211514 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:29.211514 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:29.211514 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:29.211514 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:29.212404 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:31:29.212404 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:31:29.212404 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:31:29.212404 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:31:29.212404 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 258/523] 10.222.0.89 () {56 vars in 1108 bytes} [Thu Jul 28 17:31:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:31:29.212736 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f77e4d5-767d-4910-81a8-f626094fd769 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:29.221554 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:29.254554 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbb6880-8a55-49b5-bed5-13c97e021011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:29.255292 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbb6880-8a55-49b5-bed5-13c97e021011 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:29.255642 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbb6880-8a55-49b5-bed5-13c97e021011 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:29.255989 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbb6880-8a55-49b5-bed5-13c97e021011 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:29.263873 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbb6880-8a55-49b5-bed5-13c97e021011 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:29.265615 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbb6880-8a55-49b5-bed5-13c97e021011 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:29.279999 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-ebbb6880-8a55-49b5-bed5-13c97e021011 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 28 17:31:29.279999 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:29.279999 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:29.279999 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:29.279999 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:29.279999 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:29.279999 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:29.279999 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:29.279999 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:29.279999 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:29.279999 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:29.279999 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:29.279999 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 17:31:29.279999 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 17:31:29.279999 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 17:31:29.279999 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 17:31:29.281212 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:29.281212 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:29.281212 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 17:31:29.281212 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:31:29.281212 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 17:31:29.281212 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:31:29.281212 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:29.281212 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:29.281212 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:29.281212 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:29.281212 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:29.281212 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:29.281212 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:29.281212 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:29.281212 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:29.281212 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:29.281212 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:29.281212 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:29.282549 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:29.282549 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:29.282549 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 17:31:29.282549 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 17:31:29.282549 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 17:31:29.282549 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 17:31:29.282549 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 17:31:29.282549 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 17:31:29.282549 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 28 17:31:29.282549 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:31:29.285869 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 266/524] 10.222.0.89 () {56 vars in 1102 bytes} [Thu Jul 28 17:31:29 2022] GET /identity/v3/users/neutron => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:31:29.286706 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbb6880-8a55-49b5-bed5-13c97e021011 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:29.316930 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:29.325039 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f77e4d5-767d-4910-81a8-f626094fd769 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:29.369814 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0e856c-7182-4d9e-87ae-758680a76ce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:29.370470 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0e856c-7182-4d9e-87ae-758680a76ce9 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:29.370749 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0e856c-7182-4d9e-87ae-758680a76ce9 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:29.371026 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0e856c-7182-4d9e-87ae-758680a76ce9 None admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:29.371723 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0e856c-7182-4d9e-87ae-758680a76ce9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:29.372028 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0e856c-7182-4d9e-87ae-758680a76ce9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:29.373573 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0e856c-7182-4d9e-87ae-758680a76ce9 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:29.385458 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 259/525] 10.222.0.89 () {56 vars in 1109 bytes} [Thu Jul 28 17:31:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:29.389249 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0e856c-7182-4d9e-87ae-758680a76ce9 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:29.407276 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbb6880-8a55-49b5-bed5-13c97e021011 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:29.460255 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352dd4fd-4d7b-43bf-b274-326207746d2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:29.461133 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352dd4fd-4d7b-43bf-b274-326207746d2e None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:29.461681 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352dd4fd-4d7b-43bf-b274-326207746d2e None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:29.462149 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352dd4fd-4d7b-43bf-b274-326207746d2e None admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:29.463461 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352dd4fd-4d7b-43bf-b274-326207746d2e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:29.463982 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352dd4fd-4d7b-43bf-b274-326207746d2e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:29.466385 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352dd4fd-4d7b-43bf-b274-326207746d2e None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:29.497407 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 267/526] 10.222.0.89 () {56 vars in 1148 bytes} [Thu Jul 28 17:31:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:29.499462 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352dd4fd-4d7b-43bf-b274-326207746d2e None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:29.537378 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:29.550255 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0e856c-7182-4d9e-87ae-758680a76ce9 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:29.585323 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5122502a-2fa0-4bd1-a81e-53d84c14d4a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:29.585942 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5122502a-2fa0-4bd1-a81e-53d84c14d4a4 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:29.586242 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5122502a-2fa0-4bd1-a81e-53d84c14d4a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:29.586493 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5122502a-2fa0-4bd1-a81e-53d84c14d4a4 None admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:29.587187 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5122502a-2fa0-4bd1-a81e-53d84c14d4a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:29.587490 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5122502a-2fa0-4bd1-a81e-53d84c14d4a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:29.588861 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5122502a-2fa0-4bd1-a81e-53d84c14d4a4 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:29.601828 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 260/527] 10.222.0.89 () {56 vars in 1109 bytes} [Thu Jul 28 17:31:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:29.602649 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5122502a-2fa0-4bd1-a81e-53d84c14d4a4 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:29.623144 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:29.630120 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352dd4fd-4d7b-43bf-b274-326207746d2e None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:29.669431 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb44a80-8467-40dc-a1f9-0230d2109e31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:29.671067 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb44a80-8467-40dc-a1f9-0230d2109e31 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:29.671651 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb44a80-8467-40dc-a1f9-0230d2109e31 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:29.671925 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb44a80-8467-40dc-a1f9-0230d2109e31 None admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:29.672718 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb44a80-8467-40dc-a1f9-0230d2109e31 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:29.673029 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb44a80-8467-40dc-a1f9-0230d2109e31 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:29.674606 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb44a80-8467-40dc-a1f9-0230d2109e31 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:29.697023 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 268/528] 10.222.0.89 () {56 vars in 1139 bytes} [Thu Jul 28 17:31:29 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:29.697023 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb44a80-8467-40dc-a1f9-0230d2109e31 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:29.719449 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5122502a-2fa0-4bd1-a81e-53d84c14d4a4 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:29.756852 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728878ea-4eaf-4ebb-af0f-7ccc0f5b1f63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:29.757529 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728878ea-4eaf-4ebb-af0f-7ccc0f5b1f63 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:29.757815 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728878ea-4eaf-4ebb-af0f-7ccc0f5b1f63 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:29.758091 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728878ea-4eaf-4ebb-af0f-7ccc0f5b1f63 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:29.765962 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728878ea-4eaf-4ebb-af0f-7ccc0f5b1f63 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:29.767468 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728878ea-4eaf-4ebb-af0f-7ccc0f5b1f63 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:29.779289 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-728878ea-4eaf-4ebb-af0f-7ccc0f5b1f63 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:31:29.779289 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:29.779289 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:29.779289 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:29.779289 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:29.779289 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:29.779289 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:29.779289 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:29.779289 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:29.779289 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:29.779289 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:29.779289 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:29.779289 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 17:31:29.779289 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 17:31:29.779289 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 17:31:29.780885 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 17:31:29.780885 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:29.780885 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:29.780885 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:29.780885 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:29.780885 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:29.780885 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:29.780885 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:29.780885 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:29.780885 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:29.780885 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:29.780885 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:29.780885 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:29.780885 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:29.780885 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:29.780885 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:29.780885 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:29.781800 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 17:31:29.781800 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 17:31:29.781800 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 17:31:29.781800 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 17:31:29.781800 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 17:31:29.781800 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 17:31:29.781800 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:31:29.781800 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:31:29.785310 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 261/529] 10.222.0.89 () {56 vars in 1111 bytes} [Thu Jul 28 17:31:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:31:29.786461 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728878ea-4eaf-4ebb-af0f-7ccc0f5b1f63 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:29.790354 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:29.809605 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb44a80-8467-40dc-a1f9-0230d2109e31 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:29.852402 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e0549f-d449-4c50-90b3-67582dbfce55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:29.853350 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e0549f-d449-4c50-90b3-67582dbfce55 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:29.853847 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e0549f-d449-4c50-90b3-67582dbfce55 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:29.854318 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e0549f-d449-4c50-90b3-67582dbfce55 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:29.855276 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e0549f-d449-4c50-90b3-67582dbfce55 None admin] RBAC: Adding query filter params (role.id=0a9ea6e6d6f94955a5dc949b0746a3f1, scope.project.id=04434e8535a64044996316f1d17dc0bc, user.id=d10b49b29df94404a71f43273839c252) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:29.855733 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e0549f-d449-4c50-90b3-67582dbfce55 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:29.857343 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e0549f-d449-4c50-90b3-67582dbfce55 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:29.863622 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 269/530] 10.222.0.89 () {56 vars in 1374 bytes} [Thu Jul 28 17:31:29 2022] GET /identity/v3/role_assignments?user.id=d10b49b29df94404a71f43273839c252&scope.project.id=04434e8535a64044996316f1d17dc0bc&role.id=0a9ea6e6d6f94955a5dc949b0746a3f1 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:29.864679 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e0549f-d449-4c50-90b3-67582dbfce55 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:29.892265 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:29.920851 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728878ea-4eaf-4ebb-af0f-7ccc0f5b1f63 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:29.959568 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50d3d0d-0964-4675-ad84-77989ce5b274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:29.960224 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50d3d0d-0964-4675-ad84-77989ce5b274 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:29.960506 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50d3d0d-0964-4675-ad84-77989ce5b274 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:29.960775 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50d3d0d-0964-4675-ad84-77989ce5b274 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:29.970558 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50d3d0d-0964-4675-ad84-77989ce5b274 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:29.972050 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50d3d0d-0964-4675-ad84-77989ce5b274 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:29.985119 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-a50d3d0d-0964-4675-ad84-77989ce5b274 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:31:29.985119 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:29.985119 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:29.985119 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:29.985119 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:29.985119 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:29.985119 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:29.985119 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:29.985119 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:29.985119 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:29.985119 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:29.985119 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:29.985119 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 17:31:29.985119 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 17:31:29.985119 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 17:31:29.986073 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 17:31:29.986073 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:29.986073 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:29.986073 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:29.986073 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:29.986073 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:29.986073 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:29.986073 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:29.986073 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:29.986073 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:29.986073 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:29.986073 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:29.986073 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:29.986073 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:29.986073 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:29.986073 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:29.986073 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:29.986958 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 17:31:29.986958 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 17:31:29.986958 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 17:31:29.986958 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 17:31:29.986958 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 17:31:29.986958 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 17:31:29.986958 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:31:29.986958 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:31:29.991326 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 262/531] 10.222.0.89 () {56 vars in 1111 bytes} [Thu Jul 28 17:31:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:31:29.992262 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50d3d0d-0964-4675-ad84-77989ce5b274 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:30.017893 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e0549f-d449-4c50-90b3-67582dbfce55 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:30.057844 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35829b5b-19d3-4743-818f-3ec4bc825617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:30.059277 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35829b5b-19d3-4743-818f-3ec4bc825617 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:30.059573 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35829b5b-19d3-4743-818f-3ec4bc825617 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:30.059848 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35829b5b-19d3-4743-818f-3ec4bc825617 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:30.097536 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:30.133187 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50d3d0d-0964-4675-ad84-77989ce5b274 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:30.138180 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35829b5b-19d3-4743-818f-3ec4bc825617 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:30.140353 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35829b5b-19d3-4743-818f-3ec4bc825617 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:30.182949 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c2acbe-8790-47ed-a5fa-35d3460a9be1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:30.183809 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c2acbe-8790-47ed-a5fa-35d3460a9be1 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:30.184238 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c2acbe-8790-47ed-a5fa-35d3460a9be1 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:30.184656 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c2acbe-8790-47ed-a5fa-35d3460a9be1 None admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:30.186664 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c2acbe-8790-47ed-a5fa-35d3460a9be1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:30.187147 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c2acbe-8790-47ed-a5fa-35d3460a9be1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:30.189516 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c2acbe-8790-47ed-a5fa-35d3460a9be1 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:30.195814 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:30.216049 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 263/532] 10.222.0.89 () {56 vars in 1148 bytes} [Thu Jul 28 17:31:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:30.217584 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c2acbe-8790-47ed-a5fa-35d3460a9be1 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:30.264651 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 270/533] 10.222.0.89 () {58 vars in 1282 bytes} [Thu Jul 28 17:31:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 17:31:30.266023 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35829b5b-19d3-4743-818f-3ec4bc825617 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:30.321554 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:30.345448 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c2acbe-8790-47ed-a5fa-35d3460a9be1 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:30.382881 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d05b60-a07b-497f-9a72-a1d4a145e409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:30.383968 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d05b60-a07b-497f-9a72-a1d4a145e409 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:30.384467 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d05b60-a07b-497f-9a72-a1d4a145e409 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:30.384980 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d05b60-a07b-497f-9a72-a1d4a145e409 None admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:30.386077 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d05b60-a07b-497f-9a72-a1d4a145e409 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:30.386560 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d05b60-a07b-497f-9a72-a1d4a145e409 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:30.387785 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35829b5b-19d3-4743-818f-3ec4bc825617 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:30.388428 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d05b60-a07b-497f-9a72-a1d4a145e409 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:30.406799 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 264/534] 10.222.0.89 () {56 vars in 1148 bytes} [Thu Jul 28 17:31:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:30.407925 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d05b60-a07b-497f-9a72-a1d4a145e409 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:30.430097 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891c418f-2454-449e-9002-2c8f7d69436f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:30.430862 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891c418f-2454-449e-9002-2c8f7d69436f None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:30.431246 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891c418f-2454-449e-9002-2c8f7d69436f None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:30.431620 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891c418f-2454-449e-9002-2c8f7d69436f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:30.433140 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891c418f-2454-449e-9002-2c8f7d69436f None admin] RBAC: Adding query filter params (role.id=0a9ea6e6d6f94955a5dc949b0746a3f1, scope.project.id=04434e8535a64044996316f1d17dc0bc, user.id=96d512d0c84240918dae6c4a3e8b7b79) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:30.433544 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891c418f-2454-449e-9002-2c8f7d69436f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:30.436163 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891c418f-2454-449e-9002-2c8f7d69436f None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:30.444332 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 271/535] 10.222.0.89 () {56 vars in 1374 bytes} [Thu Jul 28 17:31:30 2022] GET /identity/v3/role_assignments?user.id=96d512d0c84240918dae6c4a3e8b7b79&scope.project.id=04434e8535a64044996316f1d17dc0bc&role.id=0a9ea6e6d6f94955a5dc949b0746a3f1 => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:30.445900 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891c418f-2454-449e-9002-2c8f7d69436f None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:30.496906 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:30.584096 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d05b60-a07b-497f-9a72-a1d4a145e409 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:30.643358 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891c418f-2454-449e-9002-2c8f7d69436f None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:30.659296 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:30.664138 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b08ed4-0d94-4318-9464-cea0ad54800d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:30.664804 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b08ed4-0d94-4318-9464-cea0ad54800d None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:30.665083 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b08ed4-0d94-4318-9464-cea0ad54800d None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:30.665404 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b08ed4-0d94-4318-9464-cea0ad54800d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:30.685626 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b08ed4-0d94-4318-9464-cea0ad54800d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:30.687029 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5128ff-6b40-4f7d-a62b-53de65ae3399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:30.687629 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b08ed4-0d94-4318-9464-cea0ad54800d None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:30.688093 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5128ff-6b40-4f7d-a62b-53de65ae3399 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:30.688569 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5128ff-6b40-4f7d-a62b-53de65ae3399 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:30.689065 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5128ff-6b40-4f7d-a62b-53de65ae3399 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:30.690302 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5128ff-6b40-4f7d-a62b-53de65ae3399 None admin] RBAC: Adding query filter params (role.id=0a9ea6e6d6f94955a5dc949b0746a3f1, scope.project.id=04434e8535a64044996316f1d17dc0bc, user.id=1564db0771224a05b47d8e3cf778c147) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:30.690816 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5128ff-6b40-4f7d-a62b-53de65ae3399 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:30.692450 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5128ff-6b40-4f7d-a62b-53de65ae3399 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:30.699184 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 272/536] 10.222.0.89 () {56 vars in 1374 bytes} [Thu Jul 28 17:31:30 2022] GET /identity/v3/role_assignments?user.id=1564db0771224a05b47d8e3cf778c147&scope.project.id=04434e8535a64044996316f1d17dc0bc&role.id=0a9ea6e6d6f94955a5dc949b0746a3f1 => generated 259 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:30.714791 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-f4b08ed4-0d94-4318-9464-cea0ad54800d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:31:30.714791 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:30.714791 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:30.714791 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:30.714791 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:30.714791 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:30.714791 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:30.714791 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:30.714791 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:30.714791 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:30.714791 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:30.714791 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:30.714791 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:31:30.714791 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:31:30.714791 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:31:30.715691 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:31:30.715691 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:30.715691 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:30.715691 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:30.715691 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:30.715691 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:30.715691 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:30.715691 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:30.715691 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:30.715691 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:30.715691 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:30.715691 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:30.715691 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:30.715691 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:30.715691 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:30.715691 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:30.715691 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:30.716616 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:31:30.716616 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:31:30.716616 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:31:30.716616 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:31:30.716616 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 265/537] 10.222.0.89 () {56 vars in 1108 bytes} [Thu Jul 28 17:31:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:31:30.722245 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5128ff-6b40-4f7d-a62b-53de65ae3399 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:30.847831 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5128ff-6b40-4f7d-a62b-53de65ae3399 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:30.892934 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dba40bf-2f34-4ec7-913b-74dc596203aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:30.893596 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dba40bf-2f34-4ec7-913b-74dc596203aa None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:30.893877 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dba40bf-2f34-4ec7-913b-74dc596203aa None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:30.894151 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dba40bf-2f34-4ec7-913b-74dc596203aa None admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:30.894859 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dba40bf-2f34-4ec7-913b-74dc596203aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:30.895164 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dba40bf-2f34-4ec7-913b-74dc596203aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:30.896560 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dba40bf-2f34-4ec7-913b-74dc596203aa None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:30.910741 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 273/538] 10.222.0.89 () {56 vars in 1109 bytes} [Thu Jul 28 17:31:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:30.915974 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b08ed4-0d94-4318-9464-cea0ad54800d None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:31.073844 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b08ed4-0d94-4318-9464-cea0ad54800d None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:31.095465 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:31.101564 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046a8eb2-dd9d-442c-bad4-7586bc1a8bc0 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:31.102619 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046a8eb2-dd9d-442c-bad4-7586bc1a8bc0 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:31.103121 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046a8eb2-dd9d-442c-bad4-7586bc1a8bc0 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:31.105251 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 274/539] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:31:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:31:31.110787 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780e45b7-69d4-41f7-9f0b-55a071c651be None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:31.111076 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780e45b7-69d4-41f7-9f0b-55a071c651be None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:31.111348 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780e45b7-69d4-41f7-9f0b-55a071c651be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:31.142849 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9550c5d1-c2c6-4048-a432-0d0971bedb94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:31.143871 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9550c5d1-c2c6-4048-a432-0d0971bedb94 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:31.144356 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9550c5d1-c2c6-4048-a432-0d0971bedb94 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:31.144829 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9550c5d1-c2c6-4048-a432-0d0971bedb94 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:31.156592 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-780e45b7-69d4-41f7-9f0b-55a071c651be None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:31:31.158181 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-780e45b7-69d4-41f7-9f0b-55a071c651be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:31:31.159488 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9550c5d1-c2c6-4048-a432-0d0971bedb94 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:31.163216 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9550c5d1-c2c6-4048-a432-0d0971bedb94 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:31.188963 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-9550c5d1-c2c6-4048-a432-0d0971bedb94 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:31:31.188963 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:31.188963 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:31.188963 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:31.188963 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:31.188963 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:31.188963 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:31.188963 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:31.188963 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:31.188963 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:31.188963 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:31.188963 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:31.188963 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 17:31:31.188963 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 17:31:31.188963 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 17:31:31.189941 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 17:31:31.189941 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:31.189941 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:31.189941 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:31.189941 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:31.189941 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:31.189941 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:31.189941 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:31.189941 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:31.189941 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:31.189941 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:31.189941 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:31.189941 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:31.189941 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:31.189941 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:31.189941 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:31.189941 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:31.190786 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 17:31:31.190786 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 17:31:31.190786 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 17:31:31.190786 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 17:31:31.190786 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 17:31:31.190786 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 17:31:31.190786 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:31:31.190786 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:31:31.198894 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 266/540] 10.222.0.89 () {56 vars in 1111 bytes} [Thu Jul 28 17:31:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:31:31.204773 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9550c5d1-c2c6-4048-a432-0d0971bedb94 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:31.213391 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:31.287432 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 275/541] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:31:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:31:31.338570 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780e45b7-69d4-41f7-9f0b-55a071c651be None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:31.368021 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9550c5d1-c2c6-4048-a432-0d0971bedb94 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:31.409311 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3b719f-f40e-4cc6-8853-d4d1732df05c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:31.409937 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3b719f-f40e-4cc6-8853-d4d1732df05c None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:31.410216 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3b719f-f40e-4cc6-8853-d4d1732df05c None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:31.410488 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3b719f-f40e-4cc6-8853-d4d1732df05c None admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:31.411239 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3b719f-f40e-4cc6-8853-d4d1732df05c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:31.411542 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3b719f-f40e-4cc6-8853-d4d1732df05c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:31.421179 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3b719f-f40e-4cc6-8853-d4d1732df05c None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:31.427925 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 267/542] 10.222.0.89 () {56 vars in 1148 bytes} [Thu Jul 28 17:31:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:31:31.434582 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3b719f-f40e-4cc6-8853-d4d1732df05c None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:31.483416 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:31.492914 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780e45b7-69d4-41f7-9f0b-55a071c651be None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:31.539533 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57024798-6b1d-4d3f-a530-960092cc81b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:31.542308 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57024798-6b1d-4d3f-a530-960092cc81b8 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:31.542719 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57024798-6b1d-4d3f-a530-960092cc81b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:31.543100 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57024798-6b1d-4d3f-a530-960092cc81b8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:31.564310 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57024798-6b1d-4d3f-a530-960092cc81b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:31:31.575327 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-57024798-6b1d-4d3f-a530-960092cc81b8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 17:31:31.575327 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:31.575327 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:31.575327 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:31.575327 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:31.575327 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:31.575327 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:31.575327 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:31.575327 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:31.575327 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:31.575327 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:31.575327 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:31.575327 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:31:31.575327 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:31:31.575327 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:31:31.575327 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:31:31.576846 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:31:31.576846 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:31:31.576846 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:31:31.576846 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:31:31.576846 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:31.576846 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:31.576846 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:31.576846 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:31.576846 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:31.576846 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:31.576846 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:31.576846 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:31.576846 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:31.576846 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:31.576846 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:31.576846 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:31.577665 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:31.577665 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:31.577665 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:31.577665 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:31.577665 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:31:31.577665 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:31:31.577665 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:31:31.577665 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:31:31.577665 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:31:31.577665 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:31:31.577665 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 17:31:31.577665 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:31:31.578942 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 276/543] 10.222.0.89 () {56 vars in 1102 bytes} [Thu Jul 28 17:31:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:31:31.588908 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57024798-6b1d-4d3f-a530-960092cc81b8 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:31.617251 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3b719f-f40e-4cc6-8853-d4d1732df05c None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:31.659022 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:31.667615 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecd9636-a7a3-4524-9f5e-9975846dab6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:31.669089 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecd9636-a7a3-4524-9f5e-9975846dab6e None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:31.669602 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecd9636-a7a3-4524-9f5e-9975846dab6e None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:31.670134 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecd9636-a7a3-4524-9f5e-9975846dab6e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:31.671136 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecd9636-a7a3-4524-9f5e-9975846dab6e None admin] RBAC: Adding query filter params (role.id=0a9ea6e6d6f94955a5dc949b0746a3f1, scope.project.id=04434e8535a64044996316f1d17dc0bc, user.id=76817cd541294331b0023069092e8e89) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:31.671651 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecd9636-a7a3-4524-9f5e-9975846dab6e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:31.673624 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecd9636-a7a3-4524-9f5e-9975846dab6e None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:31.684130 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 268/544] 10.222.0.89 () {56 vars in 1374 bytes} [Thu Jul 28 17:31:31 2022] GET /identity/v3/role_assignments?user.id=76817cd541294331b0023069092e8e89&scope.project.id=04434e8535a64044996316f1d17dc0bc&role.id=0a9ea6e6d6f94955a5dc949b0746a3f1 => generated 259 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:31.756762 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57024798-6b1d-4d3f-a530-960092cc81b8 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:31.798033 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b754761a-be01-4b64-b9dc-b2d4ecfa6490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:31.799056 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b754761a-be01-4b64-b9dc-b2d4ecfa6490 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:31.799564 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b754761a-be01-4b64-b9dc-b2d4ecfa6490 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:31.800039 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b754761a-be01-4b64-b9dc-b2d4ecfa6490 None admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:31.801010 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b754761a-be01-4b64-b9dc-b2d4ecfa6490 None admin] RBAC: Adding query filter params (name=service) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:31.801541 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b754761a-be01-4b64-b9dc-b2d4ecfa6490 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:31.803148 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b754761a-be01-4b64-b9dc-b2d4ecfa6490 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:31.814601 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 277/545] 10.222.0.89 () {56 vars in 1103 bytes} [Thu Jul 28 17:31:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:31.820794 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecd9636-a7a3-4524-9f5e-9975846dab6e None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:31.959415 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:31.972996 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecd9636-a7a3-4524-9f5e-9975846dab6e None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:32.019277 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca024d66-a779-488b-a33b-8a69b240577b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:32.019936 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca024d66-a779-488b-a33b-8a69b240577b None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:32.020216 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca024d66-a779-488b-a33b-8a69b240577b None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:32.020549 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca024d66-a779-488b-a33b-8a69b240577b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:32.032694 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca024d66-a779-488b-a33b-8a69b240577b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:32.034201 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca024d66-a779-488b-a33b-8a69b240577b None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:32.049485 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-ca024d66-a779-488b-a33b-8a69b240577b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:31:32.049485 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:32.049485 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:32.049485 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:32.049485 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:32.049485 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:32.049485 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:32.049485 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:32.049485 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:32.049485 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:32.049485 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:32.049485 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:32.049485 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:31:32.049485 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:31:32.049485 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:31:32.050457 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:31:32.050457 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:32.050457 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:32.050457 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:32.050457 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:32.050457 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:32.050457 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:32.050457 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:32.050457 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:32.050457 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:32.050457 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:32.050457 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:32.050457 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:32.050457 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:32.050457 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:32.050457 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:32.050457 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:32.051692 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:31:32.051692 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:31:32.051692 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:31:32.051692 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:31:32.051692 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 269/546] 10.222.0.89 () {56 vars in 1108 bytes} [Thu Jul 28 17:31:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:31:32.056487 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b754761a-be01-4b64-b9dc-b2d4ecfa6490 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:32.187924 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:32.205190 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b754761a-be01-4b64-b9dc-b2d4ecfa6490 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:32.250362 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816b23ea-5d1e-409a-9e35-69871100ec37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:32.250903 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816b23ea-5d1e-409a-9e35-69871100ec37 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:32.251185 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816b23ea-5d1e-409a-9e35-69871100ec37 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:32.251461 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816b23ea-5d1e-409a-9e35-69871100ec37 None admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:32.252160 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816b23ea-5d1e-409a-9e35-69871100ec37 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:32.252463 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816b23ea-5d1e-409a-9e35-69871100ec37 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:32.253877 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816b23ea-5d1e-409a-9e35-69871100ec37 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:32.269816 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 278/547] 10.222.0.89 () {56 vars in 1109 bytes} [Thu Jul 28 17:31:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:32.274963 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca024d66-a779-488b-a33b-8a69b240577b None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:32.436396 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca024d66-a779-488b-a33b-8a69b240577b None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:32.486776 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacf718c-5cdf-4082-8676-1579f441ba74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:32.487419 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacf718c-5cdf-4082-8676-1579f441ba74 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:32.487697 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacf718c-5cdf-4082-8676-1579f441ba74 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:32.487961 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacf718c-5cdf-4082-8676-1579f441ba74 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:32.502361 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacf718c-5cdf-4082-8676-1579f441ba74 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:32.503738 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacf718c-5cdf-4082-8676-1579f441ba74 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:32.519453 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-cacf718c-5cdf-4082-8676-1579f441ba74 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 28 17:31:32.519453 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:32.519453 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:32.519453 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:32.519453 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:32.519453 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:32.519453 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:32.519453 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:32.519453 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:32.519453 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:32.519453 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:32.519453 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:32.519453 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 17:31:32.519453 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 17:31:32.519453 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 17:31:32.519453 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 17:31:32.520384 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:32.520384 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:32.520384 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 17:31:32.520384 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:31:32.520384 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 17:31:32.520384 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:31:32.520384 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:32.520384 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:32.520384 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:32.520384 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:32.520384 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:32.520384 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:32.520384 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:32.520384 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:32.520384 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:32.520384 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:32.520384 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:32.520384 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:32.522409 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:32.522409 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:32.522409 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 17:31:32.522409 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 17:31:32.522409 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 17:31:32.522409 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 17:31:32.522409 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 17:31:32.522409 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 17:31:32.522409 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 28 17:31:32.522409 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:31:32.522409 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 270/548] 10.222.0.89 () {56 vars in 1096 bytes} [Thu Jul 28 17:31:32 2022] GET /identity/v3/users/swift => generated 83 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:31:32.526513 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816b23ea-5d1e-409a-9e35-69871100ec37 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:32.558577 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f34fc4a-2d73-426f-b386-615ed33c2c97 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:32.558894 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f34fc4a-2d73-426f-b386-615ed33c2c97 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:32.559146 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f34fc4a-2d73-426f-b386-615ed33c2c97 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:32.561241 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 271/549] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:31:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:31:32.566267 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449cb23d-281b-471d-8cc3-3c8436f4b413 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:32.566552 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449cb23d-281b-471d-8cc3-3c8436f4b413 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:32.566826 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449cb23d-281b-471d-8cc3-3c8436f4b413 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:32.574379 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:32.594592 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-449cb23d-281b-471d-8cc3-3c8436f4b413 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:31:32.595832 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-449cb23d-281b-471d-8cc3-3c8436f4b413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:31:32.703194 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816b23ea-5d1e-409a-9e35-69871100ec37 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:32.735941 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 272/550] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:31:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 17:31:32.741084 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371b4a0e-c518-46c7-b290-a18cee5d9315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:32.741736 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371b4a0e-c518-46c7-b290-a18cee5d9315 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:32.742017 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371b4a0e-c518-46c7-b290-a18cee5d9315 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:32.742294 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371b4a0e-c518-46c7-b290-a18cee5d9315 None admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:32.743094 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371b4a0e-c518-46c7-b290-a18cee5d9315 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:32.743399 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371b4a0e-c518-46c7-b290-a18cee5d9315 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:32.745251 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371b4a0e-c518-46c7-b290-a18cee5d9315 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:32.763673 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 279/551] 10.222.0.89 () {56 vars in 1135 bytes} [Thu Jul 28 17:31:32 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:32.770756 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449cb23d-281b-471d-8cc3-3c8436f4b413 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:32.780517 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371b4a0e-c518-46c7-b290-a18cee5d9315 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:32.792134 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:32.920774 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371b4a0e-c518-46c7-b290-a18cee5d9315 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:32.959565 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2dfd26-1840-45fb-935e-2a51c3c56daf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:32.960225 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2dfd26-1840-45fb-935e-2a51c3c56daf None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:32.960660 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2dfd26-1840-45fb-935e-2a51c3c56daf None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:32.960997 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2dfd26-1840-45fb-935e-2a51c3c56daf None admin] PATH_INFO: `/v3/roles/service` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:32.973913 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2dfd26-1840-45fb-935e-2a51c3c56daf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:31:32.977106 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449cb23d-281b-471d-8cc3-3c8436f4b413 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:32.979089 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-be2dfd26-1840-45fb-935e-2a51c3c56daf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 17:31:32.979089 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:32.979089 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:32.979089 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:32.979089 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:32.979089 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:32.979089 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:32.979089 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:32.979089 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:32.979089 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:32.979089 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:32.979089 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:32.979089 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:31:32.979089 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:31:32.979089 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:31:32.979089 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:31:32.980075 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:31:32.980075 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:31:32.980075 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:31:32.980075 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:31:32.980075 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:32.980075 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:32.980075 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:32.980075 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:32.980075 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:32.980075 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:32.980075 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:32.980075 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:32.980075 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:32.980075 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:32.980075 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:32.980075 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:32.981406 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:32.981406 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:32.981406 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:32.981406 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:32.981406 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:31:32.981406 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:31:32.981406 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:31:32.981406 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:31:32.981406 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:31:32.981406 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:31:32.981406 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 17:31:32.981406 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:31:32.981406 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 280/552] 10.222.0.89 () {56 vars in 1102 bytes} [Thu Jul 28 17:31:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:31:32.982109 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17cbd87-36ae-4657-abc7-dac4f6667f10 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:32.982403 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17cbd87-36ae-4657-abc7-dac4f6667f10 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:32.983086 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17cbd87-36ae-4657-abc7-dac4f6667f10 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:32.985308 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 281/553] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:31:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:31:32.993750 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17cbd87-36ae-4657-abc7-dac4f6667f10 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:33.022789 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4420dcec-2b57-492c-ba99-fff5fce41c4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:33.023606 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4420dcec-2b57-492c-ba99-fff5fce41c4d None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:33.023898 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4420dcec-2b57-492c-ba99-fff5fce41c4d None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:33.024176 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4420dcec-2b57-492c-ba99-fff5fce41c4d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:33.028027 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:33.071700 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:33.115521 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17cbd87-36ae-4657-abc7-dac4f6667f10 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:33.120870 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4420dcec-2b57-492c-ba99-fff5fce41c4d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:33.122384 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4420dcec-2b57-492c-ba99-fff5fce41c4d None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:33.153971 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d7cc17-c916-4d58-88dd-e0ff067f5758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:33.155288 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d7cc17-c916-4d58-88dd-e0ff067f5758 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:33.155702 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d7cc17-c916-4d58-88dd-e0ff067f5758 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:33.156043 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d7cc17-c916-4d58-88dd-e0ff067f5758 None admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:33.156824 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d7cc17-c916-4d58-88dd-e0ff067f5758 None admin] RBAC: Adding query filter params (name=service) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:33.157219 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d7cc17-c916-4d58-88dd-e0ff067f5758 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:33.158754 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d7cc17-c916-4d58-88dd-e0ff067f5758 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:33.172808 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 282/554] 10.222.0.89 () {56 vars in 1103 bytes} [Thu Jul 28 17:31:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:31:33.174419 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e602e817-1abe-4602-a20b-5f45818e90ce None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:33.174779 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e602e817-1abe-4602-a20b-5f45818e90ce None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:33.174963 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e602e817-1abe-4602-a20b-5f45818e90ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:33.193440 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-e602e817-1abe-4602-a20b-5f45818e90ce None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:31:33.194632 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-e602e817-1abe-4602-a20b-5f45818e90ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:31:33.242542 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 273/555] 10.222.0.89 () {58 vars in 1282 bytes} [Thu Jul 28 17:31:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 472 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 17:31:33.243814 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4420dcec-2b57-492c-ba99-fff5fce41c4d None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:33.277407 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 283/556] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:31:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:31:33.278305 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e602e817-1abe-4602-a20b-5f45818e90ce None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:33.284031 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:33.349271 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:33.363254 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4420dcec-2b57-492c-ba99-fff5fce41c4d None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:33.399594 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e602e817-1abe-4602-a20b-5f45818e90ce None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:33.401595 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbe01fb-be2a-4b6b-9880-f529dd42b844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:33.402441 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbe01fb-be2a-4b6b-9880-f529dd42b844 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:33.402721 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbe01fb-be2a-4b6b-9880-f529dd42b844 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:33.402986 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbe01fb-be2a-4b6b-9880-f529dd42b844 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:33.413938 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbe01fb-be2a-4b6b-9880-f529dd42b844 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:33.415388 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbe01fb-be2a-4b6b-9880-f529dd42b844 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:33.431494 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-6fbe01fb-be2a-4b6b-9880-f529dd42b844 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:31:33.431494 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:33.431494 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:33.431494 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:33.431494 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:33.431494 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:33.431494 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:33.431494 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:33.431494 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:33.431494 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:33.431494 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:33.431494 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:33.431494 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:31:33.431494 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:31:33.431494 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:31:33.432393 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:31:33.432393 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:33.432393 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:33.432393 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:33.432393 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:33.432393 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:33.432393 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:33.432393 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:33.432393 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:33.432393 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:33.432393 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:33.432393 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:33.432393 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:33.432393 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:33.432393 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:33.432393 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:33.432393 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:33.433292 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:31:33.433292 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:31:33.433292 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:31:33.433292 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:31:33.433292 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 274/557] 10.222.0.89 () {56 vars in 1108 bytes} [Thu Jul 28 17:31:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:31:33.434122 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3c5ede-1bf9-4039-8fca-95c9a7a81454 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:33.434122 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3c5ede-1bf9-4039-8fca-95c9a7a81454 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:33.434398 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3c5ede-1bf9-4039-8fca-95c9a7a81454 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:33.440676 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 275/558] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:31:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:31:33.440676 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3c5ede-1bf9-4039-8fca-95c9a7a81454 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:33.443215 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa2f881-ac37-47f5-9a43-573573c0c418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:33.443871 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa2f881-ac37-47f5-9a43-573573c0c418 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:33.444150 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa2f881-ac37-47f5-9a43-573573c0c418 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:33.444420 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa2f881-ac37-47f5-9a43-573573c0c418 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:33.456039 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa2f881-ac37-47f5-9a43-573573c0c418 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:33.457520 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa2f881-ac37-47f5-9a43-573573c0c418 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:33.473532 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-bfa2f881-ac37-47f5-9a43-573573c0c418 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:31:33.473532 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:33.473532 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:33.473532 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:33.473532 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:33.473532 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:33.473532 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:33.473532 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:33.473532 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:33.473532 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:33.473532 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:33.473532 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:33.473532 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:31:33.473532 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:31:33.473532 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:31:33.474857 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:31:33.474857 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:33.474857 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:33.474857 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:33.474857 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:33.474857 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:33.474857 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:33.474857 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:33.474857 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:33.474857 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:33.474857 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:33.474857 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:33.474857 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:33.474857 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:33.474857 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:33.474857 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:33.474857 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:33.475737 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:31:33.475737 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:31:33.475737 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:31:33.475737 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:31:33.475737 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 284/559] 10.222.0.89 () {56 vars in 1108 bytes} [Thu Jul 28 17:31:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:31:33.475737 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa2f881-ac37-47f5-9a43-573573c0c418 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:33.538793 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:33.555658 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3c5ede-1bf9-4039-8fca-95c9a7a81454 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:33.586669 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa2f881-ac37-47f5-9a43-573573c0c418 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:33.588901 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a5c481-75be-4f66-be64-51f13b6abab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:33.589584 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a5c481-75be-4f66-be64-51f13b6abab2 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:33.589858 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a5c481-75be-4f66-be64-51f13b6abab2 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:33.590133 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a5c481-75be-4f66-be64-51f13b6abab2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:33.598280 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:33.601637 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a5c481-75be-4f66-be64-51f13b6abab2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:31:33.606981 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-a0a5c481-75be-4f66-be64-51f13b6abab2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 17:31:33.606981 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:33.606981 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:33.606981 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:33.606981 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:33.606981 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:33.606981 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:33.606981 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:33.606981 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:33.606981 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:33.606981 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:33.606981 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:33.606981 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:31:33.606981 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:31:33.606981 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:31:33.606981 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:31:33.608807 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:31:33.608807 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:31:33.608807 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:31:33.608807 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:31:33.608807 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:33.608807 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:33.608807 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:33.608807 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:33.608807 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:33.608807 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:33.608807 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:33.608807 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:33.608807 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:33.608807 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:33.608807 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:33.608807 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:33.610302 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:33.610302 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:33.610302 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:33.610302 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:33.610302 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:31:33.610302 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:31:33.610302 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:31:33.610302 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:31:33.610302 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:31:33.610302 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:31:33.610302 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 17:31:33.610302 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:31:33.613767 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 276/560] 10.222.0.89 () {56 vars in 1102 bytes} [Thu Jul 28 17:31:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:31:33.614869 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b005c19-da06-4fc6-a88d-2902ad40d729 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:33.615161 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b005c19-da06-4fc6-a88d-2902ad40d729 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:33.615438 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b005c19-da06-4fc6-a88d-2902ad40d729 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:33.627251 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bfe2da-fedf-4595-be9b-4acd3dbf9aee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:33.628128 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bfe2da-fedf-4595-be9b-4acd3dbf9aee None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:33.628407 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bfe2da-fedf-4595-be9b-4acd3dbf9aee None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:33.628677 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bfe2da-fedf-4595-be9b-4acd3dbf9aee None admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:33.629398 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bfe2da-fedf-4595-be9b-4acd3dbf9aee None admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:33.629703 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bfe2da-fedf-4595-be9b-4acd3dbf9aee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:33.631128 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bfe2da-fedf-4595-be9b-4acd3dbf9aee None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:33.633709 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-0b005c19-da06-4fc6-a88d-2902ad40d729 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:31:33.634901 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-0b005c19-da06-4fc6-a88d-2902ad40d729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:31:33.649102 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 285/561] 10.222.0.89 () {56 vars in 1109 bytes} [Thu Jul 28 17:31:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:33.649102 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bfe2da-fedf-4595-be9b-4acd3dbf9aee None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:33.721294 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 277/562] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:31:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:31:33.722447 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b005c19-da06-4fc6-a88d-2902ad40d729 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:33.756887 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bfe2da-fedf-4595-be9b-4acd3dbf9aee None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:33.792425 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b77ed79-d2b5-4152-94e4-5f5eb77deb9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:33.793396 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b77ed79-d2b5-4152-94e4-5f5eb77deb9f None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:33.794007 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b77ed79-d2b5-4152-94e4-5f5eb77deb9f None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:33.794599 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b77ed79-d2b5-4152-94e4-5f5eb77deb9f None admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:33.795867 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b77ed79-d2b5-4152-94e4-5f5eb77deb9f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:33.796329 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b77ed79-d2b5-4152-94e4-5f5eb77deb9f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:33.797740 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b77ed79-d2b5-4152-94e4-5f5eb77deb9f None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:33.809853 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 286/563] 10.222.0.89 () {56 vars in 1109 bytes} [Thu Jul 28 17:31:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:33.810648 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b77ed79-d2b5-4152-94e4-5f5eb77deb9f None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:33.821594 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:33.849091 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b005c19-da06-4fc6-a88d-2902ad40d729 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:33.853738 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:33.884871 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50439e2-35e9-4fdf-a2cd-e5066234d911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:33.885705 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50439e2-35e9-4fdf-a2cd-e5066234d911 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:33.886190 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50439e2-35e9-4fdf-a2cd-e5066234d911 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:33.886523 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50439e2-35e9-4fdf-a2cd-e5066234d911 None admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:33.887253 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50439e2-35e9-4fdf-a2cd-e5066234d911 None admin] RBAC: Adding query filter params (name=service) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:33.887562 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50439e2-35e9-4fdf-a2cd-e5066234d911 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:33.889533 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50439e2-35e9-4fdf-a2cd-e5066234d911 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:33.902461 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 278/564] 10.222.0.89 () {56 vars in 1103 bytes} [Thu Jul 28 17:31:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:33.902461 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50439e2-35e9-4fdf-a2cd-e5066234d911 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:33.937573 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b77ed79-d2b5-4152-94e4-5f5eb77deb9f None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:33.972642 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603e0bb9-e404-4a72-9818-142a35d013ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:33.973405 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603e0bb9-e404-4a72-9818-142a35d013ee None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:33.973804 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603e0bb9-e404-4a72-9818-142a35d013ee None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:33.974146 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603e0bb9-e404-4a72-9818-142a35d013ee None admin] PATH_INFO: `/v3/users/nova` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:33.982231 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603e0bb9-e404-4a72-9818-142a35d013ee None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:33.983889 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603e0bb9-e404-4a72-9818-142a35d013ee None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:33.995990 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-603e0bb9-e404-4a72-9818-142a35d013ee None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 17:31:33.995990 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:33.995990 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:33.995990 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:33.995990 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:33.995990 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:33.995990 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:33.995990 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:33.995990 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:33.995990 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:33.995990 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:33.995990 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:33.995990 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 17:31:33.995990 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 17:31:33.995990 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 17:31:33.995990 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 17:31:33.997422 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:33.997422 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:33.997422 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 17:31:33.997422 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:31:33.997422 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 17:31:33.997422 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:31:33.997422 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:33.997422 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:33.997422 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:33.997422 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:33.997422 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:33.997422 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:33.997422 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:33.997422 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:33.997422 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:33.997422 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:33.997422 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:33.997422 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:33.999270 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:33.999270 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:33.999270 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 17:31:33.999270 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 17:31:33.999270 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 17:31:33.999270 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 17:31:33.999270 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 17:31:33.999270 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 17:31:33.999270 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 17:31:33.999270 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:31:34.003436 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 287/565] 10.222.0.89 () {56 vars in 1093 bytes} [Thu Jul 28 17:31:33 2022] GET /identity/v3/users/nova => generated 82 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:31:34.004337 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603e0bb9-e404-4a72-9818-142a35d013ee None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:34.011906 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50439e2-35e9-4fdf-a2cd-e5066234d911 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:34.046201 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f620dfc1-e000-42b7-a426-050a496719f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:34.046752 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f620dfc1-e000-42b7-a426-050a496719f3 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:34.047023 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f620dfc1-e000-42b7-a426-050a496719f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:34.047289 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f620dfc1-e000-42b7-a426-050a496719f3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:34.054003 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:34.058270 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f620dfc1-e000-42b7-a426-050a496719f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:31:34.064355 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-f620dfc1-e000-42b7-a426-050a496719f3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 17:31:34.064355 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:34.064355 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:34.064355 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:34.064355 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:34.064355 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:34.064355 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:34.064355 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:34.064355 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:34.064355 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:34.064355 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:34.064355 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:34.064355 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:31:34.064355 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:31:34.064355 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:31:34.064355 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:31:34.065508 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:31:34.065508 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:31:34.065508 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:31:34.065508 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:31:34.065508 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:34.065508 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:34.065508 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:34.065508 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:34.065508 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:34.065508 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:34.065508 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:34.065508 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:34.065508 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:34.065508 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:34.065508 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:34.065508 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:34.066346 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:34.066346 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:34.066346 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:34.066346 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:34.066346 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:31:34.066346 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:31:34.066346 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:31:34.066346 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:31:34.066346 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:31:34.066346 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:31:34.066346 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 17:31:34.066346 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:31:34.066346 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 279/566] 10.222.0.89 () {56 vars in 1102 bytes} [Thu Jul 28 17:31:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:31:34.069676 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f620dfc1-e000-42b7-a426-050a496719f3 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:34.095367 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:34.105059 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603e0bb9-e404-4a72-9818-142a35d013ee None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:34.141289 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35735e80-08b7-4aa1-aa19-06e7a30d0434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:34.142017 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35735e80-08b7-4aa1-aa19-06e7a30d0434 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:34.142300 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35735e80-08b7-4aa1-aa19-06e7a30d0434 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:34.142571 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35735e80-08b7-4aa1-aa19-06e7a30d0434 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:34.149094 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35735e80-08b7-4aa1-aa19-06e7a30d0434 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:34.150555 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35735e80-08b7-4aa1-aa19-06e7a30d0434 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:34.162654 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-35735e80-08b7-4aa1-aa19-06e7a30d0434 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:31:34.162654 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:34.162654 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:34.162654 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:34.162654 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:34.162654 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:34.162654 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:34.162654 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:34.162654 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:34.162654 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:34.162654 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:34.162654 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:34.162654 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 17:31:34.162654 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 17:31:34.162654 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 17:31:34.163569 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 17:31:34.163569 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:34.163569 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:34.163569 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:34.163569 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:34.163569 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:34.163569 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:34.163569 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:34.163569 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:34.163569 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:34.163569 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:34.163569 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:34.163569 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:34.163569 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:34.163569 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:34.163569 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:34.163569 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:34.164592 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 17:31:34.164592 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 17:31:34.164592 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 17:31:34.164592 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 17:31:34.164592 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 17:31:34.164592 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 17:31:34.164592 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:31:34.164592 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:31:34.164592 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 288/567] 10.222.0.89 () {56 vars in 1111 bytes} [Thu Jul 28 17:31:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:31:34.165329 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d74a44-eb88-4b64-8e8b-5cdba8868c18 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:34.165398 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d74a44-eb88-4b64-8e8b-5cdba8868c18 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:34.165685 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d74a44-eb88-4b64-8e8b-5cdba8868c18 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:34.166908 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 289/568] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:31:34 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:31:34.167590 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d74a44-eb88-4b64-8e8b-5cdba8868c18 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:34.175681 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f620dfc1-e000-42b7-a426-050a496719f3 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:34.209993 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca6b10c-c1a0-440f-a200-35e4f413d35f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:34.210639 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca6b10c-c1a0-440f-a200-35e4f413d35f None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:34.210933 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca6b10c-c1a0-440f-a200-35e4f413d35f None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:34.211195 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca6b10c-c1a0-440f-a200-35e4f413d35f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:34.221707 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca6b10c-c1a0-440f-a200-35e4f413d35f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:34.223163 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca6b10c-c1a0-440f-a200-35e4f413d35f None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:34.237040 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-6ca6b10c-c1a0-440f-a200-35e4f413d35f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:31:34.237040 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:34.237040 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:34.237040 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:34.237040 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:34.237040 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:34.237040 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:34.237040 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:34.237040 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:34.237040 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:34.237040 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:34.237040 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:34.237040 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:31:34.237040 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:31:34.237040 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:31:34.238027 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:31:34.238027 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:34.238027 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:34.238027 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:34.238027 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:34.238027 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:34.238027 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:34.238027 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:34.238027 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:34.238027 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:34.238027 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:34.238027 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:34.238027 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:34.238027 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:34.238027 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:34.238027 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:34.238027 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:34.238906 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:31:34.238906 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:31:34.238906 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:31:34.238906 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:31:34.238906 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 280/569] 10.222.0.89 () {56 vars in 1108 bytes} [Thu Jul 28 17:31:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:31:34.239600 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca6b10c-c1a0-440f-a200-35e4f413d35f None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:34.276090 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d74a44-eb88-4b64-8e8b-5cdba8868c18 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:34.290325 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:34.310104 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b60ed8-5f37-4f98-8968-d97b0443bf40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:34.311186 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b60ed8-5f37-4f98-8968-d97b0443bf40 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:34.311587 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b60ed8-5f37-4f98-8968-d97b0443bf40 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:34.311893 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b60ed8-5f37-4f98-8968-d97b0443bf40 None admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:34.312693 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b60ed8-5f37-4f98-8968-d97b0443bf40 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:34.312998 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b60ed8-5f37-4f98-8968-d97b0443bf40 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:34.314461 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b60ed8-5f37-4f98-8968-d97b0443bf40 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:34.332165 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 290/570] 10.222.0.89 () {56 vars in 1133 bytes} [Thu Jul 28 17:31:34 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:34.333183 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b60ed8-5f37-4f98-8968-d97b0443bf40 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:34.343918 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:34.346347 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca6b10c-c1a0-440f-a200-35e4f413d35f None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:34.380270 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b03176e-7ae4-4c84-b514-4a53c1c44da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:34.380881 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b03176e-7ae4-4c84-b514-4a53c1c44da6 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:34.381193 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b03176e-7ae4-4c84-b514-4a53c1c44da6 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:34.381470 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b03176e-7ae4-4c84-b514-4a53c1c44da6 None admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:34.382182 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b03176e-7ae4-4c84-b514-4a53c1c44da6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:34.382490 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b03176e-7ae4-4c84-b514-4a53c1c44da6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:34.383851 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b03176e-7ae4-4c84-b514-4a53c1c44da6 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:34.392447 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 281/571] 10.222.0.89 () {56 vars in 1103 bytes} [Thu Jul 28 17:31:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:34.393701 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c9f000-8281-474e-a44b-34945ba0ebe3 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:34.393993 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c9f000-8281-474e-a44b-34945ba0ebe3 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:34.394264 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c9f000-8281-474e-a44b-34945ba0ebe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:34.412433 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-58c9f000-8281-474e-a44b-34945ba0ebe3 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:31:34.413703 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-58c9f000-8281-474e-a44b-34945ba0ebe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:31:34.432916 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b60ed8-5f37-4f98-8968-d97b0443bf40 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:34.467714 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59315c2e-2a45-4c63-b4af-7828c7358615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:34.468413 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59315c2e-2a45-4c63-b4af-7828c7358615 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:34.468691 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59315c2e-2a45-4c63-b4af-7828c7358615 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:34.468958 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59315c2e-2a45-4c63-b4af-7828c7358615 None admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:34.469738 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59315c2e-2a45-4c63-b4af-7828c7358615 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:34.470042 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59315c2e-2a45-4c63-b4af-7828c7358615 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:34.471478 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59315c2e-2a45-4c63-b4af-7828c7358615 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:34.480205 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 282/572] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:31:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:31:34.481079 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c9f000-8281-474e-a44b-34945ba0ebe3 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:34.484749 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 291/573] 10.222.0.89 () {56 vars in 1148 bytes} [Thu Jul 28 17:31:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:34.485214 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59315c2e-2a45-4c63-b4af-7828c7358615 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:34.527634 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:34.552451 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:34.585468 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c9f000-8281-474e-a44b-34945ba0ebe3 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:34.587523 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59315c2e-2a45-4c63-b4af-7828c7358615 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:34.619109 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c12f94e-4b92-4e5f-a8b1-a4e611c4318d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:34.620304 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c12f94e-4b92-4e5f-a8b1-a4e611c4318d None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:34.620462 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c12f94e-4b92-4e5f-a8b1-a4e611c4318d None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:34.620637 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c12f94e-4b92-4e5f-a8b1-a4e611c4318d None admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:34.621359 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c12f94e-4b92-4e5f-a8b1-a4e611c4318d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:34.621663 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c12f94e-4b92-4e5f-a8b1-a4e611c4318d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:34.622135 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b975751-6ad1-4045-a9b8-56b4cfc1685b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:34.622796 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b975751-6ad1-4045-a9b8-56b4cfc1685b None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:34.623049 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c12f94e-4b92-4e5f-a8b1-a4e611c4318d None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:34.623160 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b975751-6ad1-4045-a9b8-56b4cfc1685b None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:34.623344 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b975751-6ad1-4045-a9b8-56b4cfc1685b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:34.633942 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 283/574] 10.222.0.89 () {56 vars in 1109 bytes} [Thu Jul 28 17:31:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:34.634088 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b975751-6ad1-4045-a9b8-56b4cfc1685b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:34.634724 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c12f94e-4b92-4e5f-a8b1-a4e611c4318d None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:34.635801 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b975751-6ad1-4045-a9b8-56b4cfc1685b None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:34.652011 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-1b975751-6ad1-4045-a9b8-56b4cfc1685b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:31:34.652011 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:34.652011 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:34.652011 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:34.652011 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:34.652011 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:34.652011 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:34.652011 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:34.652011 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:34.652011 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:34.652011 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:34.652011 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:34.652011 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:31:34.652011 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:31:34.652011 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:31:34.653001 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:31:34.653001 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:34.653001 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:34.653001 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:34.653001 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:34.653001 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:34.653001 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:34.653001 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:34.653001 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:34.653001 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:34.653001 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:34.653001 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:34.653001 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:34.653001 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:34.653001 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:34.653001 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:34.653001 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:34.653941 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:31:34.653941 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:31:34.653941 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:31:34.653941 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:31:34.653941 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 292/575] 10.222.0.89 () {56 vars in 1108 bytes} [Thu Jul 28 17:31:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:31:34.655015 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b975751-6ad1-4045-a9b8-56b4cfc1685b None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:34.740964 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c12f94e-4b92-4e5f-a8b1-a4e611c4318d None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:34.742503 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:34.755850 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b975751-6ad1-4045-a9b8-56b4cfc1685b None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:34.775503 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:34.776336 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357c94eb-663d-47f2-b55f-eaf76b633f92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:34.776967 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357c94eb-663d-47f2-b55f-eaf76b633f92 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:34.777274 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357c94eb-663d-47f2-b55f-eaf76b633f92 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:34.777544 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357c94eb-663d-47f2-b55f-eaf76b633f92 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:34.788388 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357c94eb-663d-47f2-b55f-eaf76b633f92 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:34.789851 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357c94eb-663d-47f2-b55f-eaf76b633f92 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:34.790727 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7c1138-c2dd-492c-a136-65f237f346ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:34.791330 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7c1138-c2dd-492c-a136-65f237f346ec None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:34.791606 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7c1138-c2dd-492c-a136-65f237f346ec None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:34.791873 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7c1138-c2dd-492c-a136-65f237f346ec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:34.792648 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7c1138-c2dd-492c-a136-65f237f346ec None admin] RBAC: Adding query filter params (role.id=0a9ea6e6d6f94955a5dc949b0746a3f1, scope.project.id=04434e8535a64044996316f1d17dc0bc, user.id=3c59473b01bf4433ba3b111114fc048b) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:34.792949 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7c1138-c2dd-492c-a136-65f237f346ec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:34.794371 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7c1138-c2dd-492c-a136-65f237f346ec None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:34.799979 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 293/576] 10.222.0.89 () {56 vars in 1374 bytes} [Thu Jul 28 17:31:34 2022] GET /identity/v3/role_assignments?user.id=3c59473b01bf4433ba3b111114fc048b&scope.project.id=04434e8535a64044996316f1d17dc0bc&role.id=0a9ea6e6d6f94955a5dc949b0746a3f1 => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:34.800741 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7c1138-c2dd-492c-a136-65f237f346ec None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:34.804250 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-357c94eb-663d-47f2-b55f-eaf76b633f92 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:31:34.804250 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:34.804250 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:34.804250 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:34.804250 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:34.804250 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:34.804250 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:34.804250 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:34.804250 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:34.804250 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:34.804250 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:34.804250 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:34.804250 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:31:34.804250 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:31:34.804250 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:31:34.805192 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:31:34.805192 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:34.805192 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:34.805192 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:34.805192 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:34.805192 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:34.805192 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:34.805192 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:34.805192 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:34.805192 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:34.805192 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:34.805192 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:34.805192 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:34.805192 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:34.805192 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:34.805192 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:34.805192 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:34.806117 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:31:34.806117 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:31:34.806117 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:31:34.806117 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:31:34.806117 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 284/577] 10.222.0.89 () {56 vars in 1108 bytes} [Thu Jul 28 17:31:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:31:34.806403 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357c94eb-663d-47f2-b55f-eaf76b633f92 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:34.924540 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357c94eb-663d-47f2-b55f-eaf76b633f92 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:34.938835 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7c1138-c2dd-492c-a136-65f237f346ec None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:34.967275 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fefc1d-7dbf-4a1e-b235-84a816283d79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:34.967927 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fefc1d-7dbf-4a1e-b235-84a816283d79 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:34.968193 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fefc1d-7dbf-4a1e-b235-84a816283d79 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:34.968484 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fefc1d-7dbf-4a1e-b235-84a816283d79 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:34.975530 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a185d88-7441-4792-a4f0-5a177f7e2b8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:34.976245 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a185d88-7441-4792-a4f0-5a177f7e2b8a None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:34.976456 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fefc1d-7dbf-4a1e-b235-84a816283d79 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:34.976577 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a185d88-7441-4792-a4f0-5a177f7e2b8a None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:34.976778 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a185d88-7441-4792-a4f0-5a177f7e2b8a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:34.977970 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fefc1d-7dbf-4a1e-b235-84a816283d79 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:34.988185 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a185d88-7441-4792-a4f0-5a177f7e2b8a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:31:34.989562 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-a3fefc1d-7dbf-4a1e-b235-84a816283d79 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 28 17:31:34.989562 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:34.989562 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:34.989562 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:34.989562 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:34.989562 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:34.989562 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:34.989562 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:34.989562 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:34.989562 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:34.989562 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:34.989562 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:34.989562 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 17:31:34.989562 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 17:31:34.989562 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 17:31:34.989562 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 17:31:34.990654 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:34.990654 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:34.990654 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 17:31:34.990654 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:31:34.990654 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 17:31:34.990654 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:31:34.990654 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:34.990654 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:34.990654 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:34.990654 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:34.990654 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:34.990654 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:34.990654 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:34.990654 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:34.990654 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:34.990654 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:34.990654 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:34.990654 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:34.991650 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:34.991650 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:34.991650 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 17:31:34.991650 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 17:31:34.991650 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 17:31:34.991650 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 17:31:34.991650 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 17:31:34.991650 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 17:31:34.991650 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 28 17:31:34.991650 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:31:34.991650 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 285/578] 10.222.0.89 () {56 vars in 1099 bytes} [Thu Jul 28 17:31:34 2022] GET /identity/v3/users/cinder => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:31:34.991650 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fefc1d-7dbf-4a1e-b235-84a816283d79 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:34.994204 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-8a185d88-7441-4792-a4f0-5a177f7e2b8a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 17:31:34.994204 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:34.994204 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:34.994204 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:34.994204 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:34.994204 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:34.994204 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:34.994204 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:34.994204 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:34.994204 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:34.994204 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:34.994204 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:34.994204 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:31:34.994204 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:31:34.994204 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:31:34.994204 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:31:34.995400 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:31:34.995400 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:31:34.995400 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:31:34.995400 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:31:34.995400 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:34.995400 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:34.995400 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:34.995400 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:34.995400 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:34.995400 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:34.995400 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:34.995400 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:34.995400 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:34.995400 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:34.995400 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:34.995400 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:34.996259 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:34.996259 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:34.996259 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:34.996259 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:34.996259 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:31:34.996259 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:31:34.996259 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:31:34.996259 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:31:34.996259 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:31:34.996259 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:31:34.996259 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 17:31:34.996259 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:31:35.001184 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 294/579] 10.222.0.89 () {56 vars in 1102 bytes} [Thu Jul 28 17:31:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:31:35.002051 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a185d88-7441-4792-a4f0-5a177f7e2b8a None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:35.015953 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:35.051666 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:35.112124 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a185d88-7441-4792-a4f0-5a177f7e2b8a None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:35.112326 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fefc1d-7dbf-4a1e-b235-84a816283d79 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:35.149214 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da38cd7-eb4f-4022-85f0-c5227711d7ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:35.149831 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da38cd7-eb4f-4022-85f0-c5227711d7ff None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:35.150223 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91006338-202d-4a6b-a78d-70f98cf75ead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:35.150364 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da38cd7-eb4f-4022-85f0-c5227711d7ff None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:35.150704 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da38cd7-eb4f-4022-85f0-c5227711d7ff None admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:35.150865 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91006338-202d-4a6b-a78d-70f98cf75ead None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:35.151317 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91006338-202d-4a6b-a78d-70f98cf75ead None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:35.151494 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da38cd7-eb4f-4022-85f0-c5227711d7ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:35.151771 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91006338-202d-4a6b-a78d-70f98cf75ead None admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:35.151868 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da38cd7-eb4f-4022-85f0-c5227711d7ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:35.152472 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91006338-202d-4a6b-a78d-70f98cf75ead None admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:35.152781 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91006338-202d-4a6b-a78d-70f98cf75ead None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:35.153167 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da38cd7-eb4f-4022-85f0-c5227711d7ff None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:35.154197 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91006338-202d-4a6b-a78d-70f98cf75ead None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:35.166436 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 286/580] 10.222.0.89 () {56 vars in 1109 bytes} [Thu Jul 28 17:31:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:35.167197 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 295/581] 10.222.0.89 () {56 vars in 1109 bytes} [Thu Jul 28 17:31:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:35.167918 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91006338-202d-4a6b-a78d-70f98cf75ead None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:35.175524 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da38cd7-eb4f-4022-85f0-c5227711d7ff None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:35.258142 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:35.275122 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91006338-202d-4a6b-a78d-70f98cf75ead None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:35.278417 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:35.292924 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da38cd7-eb4f-4022-85f0-c5227711d7ff None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:35.315421 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367f2992-715e-4205-bb41-8ec30af274b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:35.316022 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367f2992-715e-4205-bb41-8ec30af274b6 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:35.316299 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367f2992-715e-4205-bb41-8ec30af274b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:35.316570 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367f2992-715e-4205-bb41-8ec30af274b6 None admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:35.317305 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367f2992-715e-4205-bb41-8ec30af274b6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:35.317604 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367f2992-715e-4205-bb41-8ec30af274b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:35.318969 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367f2992-715e-4205-bb41-8ec30af274b6 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:35.328277 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 296/582] 10.222.0.89 () {56 vars in 1103 bytes} [Thu Jul 28 17:31:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:35.330314 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367f2992-715e-4205-bb41-8ec30af274b6 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:35.331139 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dca4f9f-8ba6-4221-9f5c-c8fc297e6042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:35.331966 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dca4f9f-8ba6-4221-9f5c-c8fc297e6042 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:35.332397 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dca4f9f-8ba6-4221-9f5c-c8fc297e6042 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:35.332769 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dca4f9f-8ba6-4221-9f5c-c8fc297e6042 None admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:35.333734 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dca4f9f-8ba6-4221-9f5c-c8fc297e6042 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:35.334138 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dca4f9f-8ba6-4221-9f5c-c8fc297e6042 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:35.335691 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dca4f9f-8ba6-4221-9f5c-c8fc297e6042 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:35.381580 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 287/583] 10.222.0.89 () {56 vars in 1137 bytes} [Thu Jul 28 17:31:35 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:35.382453 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dca4f9f-8ba6-4221-9f5c-c8fc297e6042 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:35.446333 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367f2992-715e-4205-bb41-8ec30af274b6 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:35.487570 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fda58f-c983-46d6-8b61-4b16ff682e80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:35.488215 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fda58f-c983-46d6-8b61-4b16ff682e80 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:35.488497 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fda58f-c983-46d6-8b61-4b16ff682e80 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:35.488769 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fda58f-c983-46d6-8b61-4b16ff682e80 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:35.495206 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dca4f9f-8ba6-4221-9f5c-c8fc297e6042 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:35.496278 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fda58f-c983-46d6-8b61-4b16ff682e80 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:35.497765 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fda58f-c983-46d6-8b61-4b16ff682e80 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:35.509993 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-27fda58f-c983-46d6-8b61-4b16ff682e80 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:31:35.509993 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:35.509993 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:35.509993 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:35.509993 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:35.509993 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:35.509993 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:35.509993 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:35.509993 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:35.509993 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:35.509993 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:35.509993 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:35.509993 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 17:31:35.509993 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 17:31:35.509993 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 17:31:35.511101 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 17:31:35.511101 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:35.511101 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:35.511101 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:35.511101 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:35.511101 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:35.511101 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:35.511101 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:35.511101 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:35.511101 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:35.511101 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:35.511101 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:35.511101 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:35.511101 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:35.511101 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:35.511101 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:35.511101 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:35.512300 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 17:31:35.512300 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 17:31:35.512300 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 17:31:35.512300 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 17:31:35.512300 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 17:31:35.512300 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 17:31:35.512300 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:31:35.512300 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:31:35.516465 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 297/584] 10.222.0.89 () {56 vars in 1111 bytes} [Thu Jul 28 17:31:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:31:35.517200 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fda58f-c983-46d6-8b61-4b16ff682e80 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:35.534666 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:35.541627 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712a4376-a82f-4ffc-90c7-7a3c3c51ca73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:35.542412 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712a4376-a82f-4ffc-90c7-7a3c3c51ca73 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:35.542704 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712a4376-a82f-4ffc-90c7-7a3c3c51ca73 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:35.542987 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712a4376-a82f-4ffc-90c7-7a3c3c51ca73 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:35.546082 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:35.551168 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712a4376-a82f-4ffc-90c7-7a3c3c51ca73 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:35.552632 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712a4376-a82f-4ffc-90c7-7a3c3c51ca73 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:35.566493 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-712a4376-a82f-4ffc-90c7-7a3c3c51ca73 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 28 17:31:35.566493 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:35.566493 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:35.566493 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:35.566493 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:35.566493 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:35.566493 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:35.566493 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:35.566493 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:35.566493 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:35.566493 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:35.566493 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:35.566493 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 17:31:35.566493 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 17:31:35.566493 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 17:31:35.566493 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 17:31:35.567667 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:35.567667 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:35.567667 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 17:31:35.567667 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:31:35.567667 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 17:31:35.567667 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:31:35.567667 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:35.567667 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:35.567667 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:35.567667 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:35.567667 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:35.567667 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:35.567667 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:35.567667 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:35.567667 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:35.567667 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:35.567667 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:35.567667 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:35.568746 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:35.568746 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:35.568746 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 17:31:35.568746 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 17:31:35.568746 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 17:31:35.568746 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 17:31:35.568746 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 17:31:35.568746 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 17:31:35.568746 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 28 17:31:35.568746 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:31:35.569814 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 288/585] 10.222.0.89 () {56 vars in 1099 bytes} [Thu Jul 28 17:31:35 2022] GET /identity/v3/users/glance => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:31:35.570406 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712a4376-a82f-4ffc-90c7-7a3c3c51ca73 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:35.634516 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fda58f-c983-46d6-8b61-4b16ff682e80 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:35.672649 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b591ad7-2cf0-47f6-b30e-9ff5b081069a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:35.673319 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b591ad7-2cf0-47f6-b30e-9ff5b081069a None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:35.673609 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b591ad7-2cf0-47f6-b30e-9ff5b081069a None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:35.673868 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b591ad7-2cf0-47f6-b30e-9ff5b081069a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:35.677021 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712a4376-a82f-4ffc-90c7-7a3c3c51ca73 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:35.685075 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b591ad7-2cf0-47f6-b30e-9ff5b081069a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:35.686684 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b591ad7-2cf0-47f6-b30e-9ff5b081069a None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:35.701308 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-3b591ad7-2cf0-47f6-b30e-9ff5b081069a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:31:35.701308 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:35.701308 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:35.701308 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:35.701308 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:35.701308 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:35.701308 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:35.701308 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:35.701308 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:35.701308 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:35.701308 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:35.701308 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:35.701308 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:31:35.701308 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:31:35.701308 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:31:35.702309 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:31:35.702309 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:35.702309 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:35.702309 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:35.702309 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:35.702309 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:35.702309 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:35.702309 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:35.702309 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:35.702309 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:35.702309 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:35.702309 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:35.702309 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:35.702309 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:35.702309 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:35.702309 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:35.702309 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:35.703208 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:31:35.703208 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:31:35.703208 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:31:35.703208 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:31:35.703208 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 298/586] 10.222.0.89 () {56 vars in 1108 bytes} [Thu Jul 28 17:31:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:31:35.703208 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b591ad7-2cf0-47f6-b30e-9ff5b081069a None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:35.716736 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e383082-7960-4c7e-a809-2ad492784a0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:35.718191 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e383082-7960-4c7e-a809-2ad492784a0a None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:35.718315 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e383082-7960-4c7e-a809-2ad492784a0a None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:35.718591 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e383082-7960-4c7e-a809-2ad492784a0a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:35.729484 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e383082-7960-4c7e-a809-2ad492784a0a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:35.731026 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e383082-7960-4c7e-a809-2ad492784a0a None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:35.745924 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-2e383082-7960-4c7e-a809-2ad492784a0a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:31:35.745924 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:35.745924 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:35.745924 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:35.745924 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:35.745924 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:35.745924 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:35.745924 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:35.745924 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:35.745924 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:35.745924 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:35.745924 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:35.745924 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:31:35.745924 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:31:35.745924 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:31:35.746826 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:31:35.746826 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:35.746826 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:35.746826 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:35.746826 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:35.746826 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:35.746826 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:35.746826 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:35.746826 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:35.746826 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:35.746826 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:35.746826 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:35.746826 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:35.746826 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:35.746826 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:35.746826 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:35.746826 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:35.747729 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:31:35.747729 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:31:35.747729 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:31:35.747729 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:31:35.747729 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 289/587] 10.222.0.89 () {56 vars in 1108 bytes} [Thu Jul 28 17:31:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:31:35.748535 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e383082-7960-4c7e-a809-2ad492784a0a None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:35.794583 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:35.799584 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:35.832932 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b591ad7-2cf0-47f6-b30e-9ff5b081069a None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:35.856968 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e383082-7960-4c7e-a809-2ad492784a0a None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:35.870546 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6049e4-310d-4a53-821b-91d29f091040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:35.871161 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6049e4-310d-4a53-821b-91d29f091040 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:35.871437 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6049e4-310d-4a53-821b-91d29f091040 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:35.871705 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6049e4-310d-4a53-821b-91d29f091040 None admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:35.872449 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6049e4-310d-4a53-821b-91d29f091040 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:35.872753 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6049e4-310d-4a53-821b-91d29f091040 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:35.875364 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6049e4-310d-4a53-821b-91d29f091040 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:35.888349 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 299/588] 10.222.0.89 () {56 vars in 1148 bytes} [Thu Jul 28 17:31:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:35.889594 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6049e4-310d-4a53-821b-91d29f091040 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:35.901330 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773f45fc-e814-4ed9-9044-41f8c9af2fba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:35.901967 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773f45fc-e814-4ed9-9044-41f8c9af2fba None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:35.902681 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773f45fc-e814-4ed9-9044-41f8c9af2fba None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:35.902965 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773f45fc-e814-4ed9-9044-41f8c9af2fba None admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:35.903939 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773f45fc-e814-4ed9-9044-41f8c9af2fba None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:35.904258 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773f45fc-e814-4ed9-9044-41f8c9af2fba None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:35.905766 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773f45fc-e814-4ed9-9044-41f8c9af2fba None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:35.925543 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 290/589] 10.222.0.89 () {56 vars in 1137 bytes} [Thu Jul 28 17:31:35 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:35.931038 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773f45fc-e814-4ed9-9044-41f8c9af2fba None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:36.007922 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6049e4-310d-4a53-821b-91d29f091040 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:36.036980 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773f45fc-e814-4ed9-9044-41f8c9af2fba None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:36.038773 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:36.043587 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663a86ad-bdc9-445e-97e2-3651fecd7c15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:36.044203 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663a86ad-bdc9-445e-97e2-3651fecd7c15 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:36.044482 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663a86ad-bdc9-445e-97e2-3651fecd7c15 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:36.044758 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663a86ad-bdc9-445e-97e2-3651fecd7c15 None admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:36.046211 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663a86ad-bdc9-445e-97e2-3651fecd7c15 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:36.046518 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663a86ad-bdc9-445e-97e2-3651fecd7c15 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:36.047888 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663a86ad-bdc9-445e-97e2-3651fecd7c15 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:36.054585 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:36.065034 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 300/590] 10.222.0.89 () {56 vars in 1109 bytes} [Thu Jul 28 17:31:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:36.065034 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663a86ad-bdc9-445e-97e2-3651fecd7c15 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:36.080917 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c784451c-173e-4e0c-8ca8-cb80f947a7f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:36.081574 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c784451c-173e-4e0c-8ca8-cb80f947a7f1 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:36.081858 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c784451c-173e-4e0c-8ca8-cb80f947a7f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:36.082138 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c784451c-173e-4e0c-8ca8-cb80f947a7f1 None admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:36.082840 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c784451c-173e-4e0c-8ca8-cb80f947a7f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:36.083144 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c784451c-173e-4e0c-8ca8-cb80f947a7f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:36.084526 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c784451c-173e-4e0c-8ca8-cb80f947a7f1 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:36.096412 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 291/591] 10.222.0.89 () {56 vars in 1109 bytes} [Thu Jul 28 17:31:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:36.097246 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c784451c-173e-4e0c-8ca8-cb80f947a7f1 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:36.177138 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663a86ad-bdc9-445e-97e2-3651fecd7c15 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:36.209513 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c784451c-173e-4e0c-8ca8-cb80f947a7f1 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:36.214250 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700168f9-f8a3-43ef-9fd6-89a0b7804697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:36.214910 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700168f9-f8a3-43ef-9fd6-89a0b7804697 None admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:36.215186 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700168f9-f8a3-43ef-9fd6-89a0b7804697 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:36.215457 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700168f9-f8a3-43ef-9fd6-89a0b7804697 None admin] PATH_INFO: `/v3/projects/04434e8535a64044996316f1d17dc0bc/users/d10b49b29df94404a71f43273839c252/roles/0a9ea6e6d6f94955a5dc949b0746a3f1` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:36.231209 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700168f9-f8a3-43ef-9fd6-89a0b7804697 None admin] RBAC: Authorizing `identity:create_grant(project_id=04434e8535a64044996316f1d17dc0bc, user_id=d10b49b29df94404a71f43273839c252, role_id=0a9ea6e6d6f94955a5dc949b0746a3f1)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:36.233749 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700168f9-f8a3-43ef-9fd6-89a0b7804697 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:36.246852 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 301/592] 10.222.0.89 () {58 vars in 1439 bytes} [Thu Jul 28 17:31:36 2022] PUT /identity/v3/projects/04434e8535a64044996316f1d17dc0bc/users/d10b49b29df94404a71f43273839c252/roles/0a9ea6e6d6f94955a5dc949b0746a3f1 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:31:36.247459 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700168f9-f8a3-43ef-9fd6-89a0b7804697 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:36.258831 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6a0932-a12e-43ef-b7bd-7ce50125e2e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:36.259485 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6a0932-a12e-43ef-b7bd-7ce50125e2e9 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:36.259767 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6a0932-a12e-43ef-b7bd-7ce50125e2e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:36.260199 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6a0932-a12e-43ef-b7bd-7ce50125e2e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:36.278601 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6a0932-a12e-43ef-b7bd-7ce50125e2e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:36.281895 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6a0932-a12e-43ef-b7bd-7ce50125e2e9 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:36.306088 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-1b6a0932-a12e-43ef-b7bd-7ce50125e2e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:31:36.306088 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:36.306088 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:36.306088 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:36.306088 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:36.306088 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:36.306088 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:36.306088 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:36.306088 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:36.306088 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:36.306088 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:36.306088 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:36.306088 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:31:36.306088 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:31:36.306088 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:31:36.307262 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:31:36.307262 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:36.307262 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:36.307262 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:36.307262 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:36.307262 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:36.307262 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:36.307262 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:36.307262 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:36.307262 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:36.307262 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:36.307262 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:36.307262 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:36.307262 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:36.307262 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:36.307262 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:36.307262 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:36.308174 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:31:36.308174 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:31:36.308174 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:31:36.308174 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:31:36.315067 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 292/593] 10.222.0.89 () {56 vars in 1108 bytes} [Thu Jul 28 17:31:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:31:36.316006 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:36.316119 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6a0932-a12e-43ef-b7bd-7ce50125e2e9 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:36.343915 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:36.381859 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700168f9-f8a3-43ef-9fd6-89a0b7804697 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:36.423186 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cf86c0-4229-4f08-9234-40963456e263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:36.423824 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cf86c0-4229-4f08-9234-40963456e263 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:36.424107 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cf86c0-4229-4f08-9234-40963456e263 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:36.424437 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cf86c0-4229-4f08-9234-40963456e263 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:36.432377 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cf86c0-4229-4f08-9234-40963456e263 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:36.433864 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cf86c0-4229-4f08-9234-40963456e263 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:36.448164 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-54cf86c0-4229-4f08-9234-40963456e263 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 28 17:31:36.448164 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:36.448164 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:36.448164 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:36.448164 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:36.448164 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:36.448164 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:36.448164 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:36.448164 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:36.448164 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:36.448164 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:36.448164 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:36.448164 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 17:31:36.448164 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 17:31:36.448164 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 17:31:36.448164 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 17:31:36.449131 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:36.449131 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:36.449131 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 17:31:36.449131 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:31:36.449131 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 17:31:36.449131 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:31:36.449131 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:36.449131 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:36.449131 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:36.449131 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:36.449131 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:36.449131 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:36.449131 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:36.449131 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:36.449131 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:36.449131 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:36.449131 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:36.449131 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:36.450117 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:36.450117 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:36.450117 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 17:31:36.450117 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 17:31:36.450117 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 17:31:36.450117 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 17:31:36.450117 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 17:31:36.450117 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 17:31:36.450117 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 28 17:31:36.450117 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:31:36.450117 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 302/594] 10.222.0.89 () {56 vars in 1102 bytes} [Thu Jul 28 17:31:36 2022] GET /identity/v3/users/neutron => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:31:36.450117 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cf86c0-4229-4f08-9234-40963456e263 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:36.463128 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6a0932-a12e-43ef-b7bd-7ce50125e2e9 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:36.515946 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4fc4ad-c1ec-49dc-936e-490c06708b8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:36.516825 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4fc4ad-c1ec-49dc-936e-490c06708b8f None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:36.517106 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4fc4ad-c1ec-49dc-936e-490c06708b8f None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:36.517418 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4fc4ad-c1ec-49dc-936e-490c06708b8f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:36.527400 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4fc4ad-c1ec-49dc-936e-490c06708b8f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:36.529792 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4fc4ad-c1ec-49dc-936e-490c06708b8f None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:36.546327 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-0e4fc4ad-c1ec-49dc-936e-490c06708b8f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:31:36.546327 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:36.546327 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:36.546327 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:36.546327 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:36.546327 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:36.546327 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:36.546327 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:36.546327 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:36.546327 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:36.546327 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:36.546327 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:36.546327 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 17:31:36.546327 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 17:31:36.546327 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 17:31:36.547270 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 17:31:36.547270 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:36.547270 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:36.547270 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:36.547270 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:36.547270 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:36.547270 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:36.547270 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:36.547270 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:36.547270 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:36.547270 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:36.547270 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:36.547270 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:36.547270 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:36.547270 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:36.547270 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:36.547270 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:36.548150 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 17:31:36.548150 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 17:31:36.548150 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 17:31:36.548150 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 17:31:36.548150 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 17:31:36.548150 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 17:31:36.548150 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:31:36.548150 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:31:36.548602 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 293/595] 10.222.0.89 () {56 vars in 1111 bytes} [Thu Jul 28 17:31:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:31:36.549769 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4fc4ad-c1ec-49dc-936e-490c06708b8f None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:36.574438 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cf86c0-4229-4f08-9234-40963456e263 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:36.584658 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:36.613348 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e280f30f-75c1-4173-adb9-dd08c495b37b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:36.613742 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e280f30f-75c1-4173-adb9-dd08c495b37b None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:36.614023 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e280f30f-75c1-4173-adb9-dd08c495b37b None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:36.614300 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e280f30f-75c1-4173-adb9-dd08c495b37b None admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:36.615060 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e280f30f-75c1-4173-adb9-dd08c495b37b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:36.615287 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e280f30f-75c1-4173-adb9-dd08c495b37b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:36.616782 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e280f30f-75c1-4173-adb9-dd08c495b37b None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:36.630461 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 303/596] 10.222.0.89 () {56 vars in 1109 bytes} [Thu Jul 28 17:31:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:36.631640 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e280f30f-75c1-4173-adb9-dd08c495b37b None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:36.646172 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:36.672620 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4fc4ad-c1ec-49dc-936e-490c06708b8f None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:36.708894 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ff793c-f52c-4301-82ca-3c6555aac12c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:36.709723 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ff793c-f52c-4301-82ca-3c6555aac12c None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:36.710071 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ff793c-f52c-4301-82ca-3c6555aac12c None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:36.710415 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ff793c-f52c-4301-82ca-3c6555aac12c None admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:36.711300 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ff793c-f52c-4301-82ca-3c6555aac12c None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:36.711659 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ff793c-f52c-4301-82ca-3c6555aac12c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:36.713185 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ff793c-f52c-4301-82ca-3c6555aac12c None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:36.738557 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 294/597] 10.222.0.89 () {56 vars in 1139 bytes} [Thu Jul 28 17:31:36 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:36.738557 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ff793c-f52c-4301-82ca-3c6555aac12c None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:36.765243 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e280f30f-75c1-4173-adb9-dd08c495b37b None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:36.813899 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6357f99-a10b-4966-b870-4f467fb73cae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:36.814504 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6357f99-a10b-4966-b870-4f467fb73cae None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:36.814838 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6357f99-a10b-4966-b870-4f467fb73cae None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:36.815186 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6357f99-a10b-4966-b870-4f467fb73cae None admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:36.816135 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6357f99-a10b-4966-b870-4f467fb73cae None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:36.816513 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6357f99-a10b-4966-b870-4f467fb73cae None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:36.818167 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6357f99-a10b-4966-b870-4f467fb73cae None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:36.841944 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 304/598] 10.222.0.89 () {56 vars in 1148 bytes} [Thu Jul 28 17:31:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:36.841944 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6357f99-a10b-4966-b870-4f467fb73cae None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:36.856837 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ff793c-f52c-4301-82ca-3c6555aac12c None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:36.880542 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:36.890508 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:36.896297 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2893fefc-5486-4f03-aef8-e67446596b73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:36.896946 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2893fefc-5486-4f03-aef8-e67446596b73 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:36.897254 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2893fefc-5486-4f03-aef8-e67446596b73 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:36.897528 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2893fefc-5486-4f03-aef8-e67446596b73 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:36.905541 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2893fefc-5486-4f03-aef8-e67446596b73 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:36.907452 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2893fefc-5486-4f03-aef8-e67446596b73 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:36.919475 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-2893fefc-5486-4f03-aef8-e67446596b73 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:31:36.919475 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:36.919475 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:36.919475 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:36.919475 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:36.919475 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:36.919475 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:36.919475 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:36.919475 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:36.919475 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:36.919475 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:36.919475 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:36.919475 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 17:31:36.919475 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 17:31:36.919475 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 17:31:36.920417 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 17:31:36.920417 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:36.920417 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:36.920417 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:36.920417 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:36.920417 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:36.920417 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:36.920417 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:36.920417 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:36.920417 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:36.920417 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:36.920417 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:36.920417 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:36.920417 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:36.920417 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:36.920417 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:36.920417 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:36.921793 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 17:31:36.921793 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 17:31:36.921793 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 17:31:36.921793 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 17:31:36.921793 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 17:31:36.921793 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 17:31:36.921793 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:31:36.921793 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:31:36.925444 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 295/599] 10.222.0.89 () {56 vars in 1111 bytes} [Thu Jul 28 17:31:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:31:36.926313 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2893fefc-5486-4f03-aef8-e67446596b73 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:36.961193 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6357f99-a10b-4966-b870-4f467fb73cae None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:37.004412 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c1db6d-f732-4fe2-ad96-cbb6e0cab3b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:37.005067 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c1db6d-f732-4fe2-ad96-cbb6e0cab3b4 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:37.005378 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c1db6d-f732-4fe2-ad96-cbb6e0cab3b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:37.005654 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c1db6d-f732-4fe2-ad96-cbb6e0cab3b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:37.019886 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c1db6d-f732-4fe2-ad96-cbb6e0cab3b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:37.021411 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c1db6d-f732-4fe2-ad96-cbb6e0cab3b4 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:37.034704 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2893fefc-5486-4f03-aef8-e67446596b73 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:37.036567 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-51c1db6d-f732-4fe2-ad96-cbb6e0cab3b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:31:37.036567 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:37.036567 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:37.036567 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:37.036567 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:37.036567 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:37.036567 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:37.036567 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:37.036567 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:37.036567 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:37.036567 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:37.036567 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:37.036567 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:31:37.036567 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:31:37.036567 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:31:37.037528 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:31:37.037528 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:37.037528 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:37.037528 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:37.037528 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:37.037528 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:37.037528 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:37.037528 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:37.037528 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:37.037528 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:37.037528 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:37.037528 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:37.037528 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:37.037528 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:37.037528 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:37.037528 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:37.037528 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:37.038430 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:31:37.038430 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:31:37.038430 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:31:37.038430 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:31:37.038430 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 305/600] 10.222.0.89 () {56 vars in 1108 bytes} [Thu Jul 28 17:31:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:31:37.039806 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b2e594-d214-42df-9879-33a82ce3226c None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:37.039806 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b2e594-d214-42df-9879-33a82ce3226c None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:37.039806 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b2e594-d214-42df-9879-33a82ce3226c None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:37.040846 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 306/601] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:31:37 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:31:37.045908 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b2e594-d214-42df-9879-33a82ce3226c None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:37.075393 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635aeb2a-cf96-4d7f-b984-a76bc56be25b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:37.076085 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635aeb2a-cf96-4d7f-b984-a76bc56be25b None admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:37.076432 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635aeb2a-cf96-4d7f-b984-a76bc56be25b None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:37.076975 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635aeb2a-cf96-4d7f-b984-a76bc56be25b None admin] PATH_INFO: `/v3/projects/04434e8535a64044996316f1d17dc0bc/users/96d512d0c84240918dae6c4a3e8b7b79/roles/0a9ea6e6d6f94955a5dc949b0746a3f1` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:37.092905 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635aeb2a-cf96-4d7f-b984-a76bc56be25b None admin] RBAC: Authorizing `identity:create_grant(project_id=04434e8535a64044996316f1d17dc0bc, user_id=96d512d0c84240918dae6c4a3e8b7b79, role_id=0a9ea6e6d6f94955a5dc949b0746a3f1)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:37.094439 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635aeb2a-cf96-4d7f-b984-a76bc56be25b None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:37.108626 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 296/602] 10.222.0.89 () {58 vars in 1439 bytes} [Thu Jul 28 17:31:36 2022] PUT /identity/v3/projects/04434e8535a64044996316f1d17dc0bc/users/96d512d0c84240918dae6c4a3e8b7b79/roles/0a9ea6e6d6f94955a5dc949b0746a3f1 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:31:37.109328 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635aeb2a-cf96-4d7f-b984-a76bc56be25b None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:37.156931 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:37.174598 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:37.189549 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b2e594-d214-42df-9879-33a82ce3226c None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:37.233543 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f8234e-6db8-4786-a51c-e004ccadeb00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:37.234780 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f8234e-6db8-4786-a51c-e004ccadeb00 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:37.235305 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f8234e-6db8-4786-a51c-e004ccadeb00 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:37.235848 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f8234e-6db8-4786-a51c-e004ccadeb00 None admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:37.237053 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f8234e-6db8-4786-a51c-e004ccadeb00 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:37.237610 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f8234e-6db8-4786-a51c-e004ccadeb00 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:37.239294 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f8234e-6db8-4786-a51c-e004ccadeb00 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:37.255850 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 307/603] 10.222.0.89 () {56 vars in 1148 bytes} [Thu Jul 28 17:31:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:37.257605 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1c0611-65b8-4440-b105-9d9243fd63df None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:37.258107 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1c0611-65b8-4440-b105-9d9243fd63df None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:37.258639 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1c0611-65b8-4440-b105-9d9243fd63df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:37.261298 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635aeb2a-cf96-4d7f-b984-a76bc56be25b None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:37.281575 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-bc1c0611-65b8-4440-b105-9d9243fd63df None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:31:37.282796 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-bc1c0611-65b8-4440-b105-9d9243fd63df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:31:37.319296 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db20637-c966-4077-bdc0-9bb6e0a52c52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:37.319296 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db20637-c966-4077-bdc0-9bb6e0a52c52 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:37.319296 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db20637-c966-4077-bdc0-9bb6e0a52c52 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:37.319296 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db20637-c966-4077-bdc0-9bb6e0a52c52 None admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:37.319704 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db20637-c966-4077-bdc0-9bb6e0a52c52 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:37.319971 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db20637-c966-4077-bdc0-9bb6e0a52c52 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:37.321402 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db20637-c966-4077-bdc0-9bb6e0a52c52 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:37.340786 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 297/604] 10.222.0.89 () {56 vars in 1109 bytes} [Thu Jul 28 17:31:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:37.342549 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db20637-c966-4077-bdc0-9bb6e0a52c52 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:37.367314 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 308/605] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:31:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:31:37.374403 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1c0611-65b8-4440-b105-9d9243fd63df None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:37.449354 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db20637-c966-4077-bdc0-9bb6e0a52c52 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:37.451758 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:37.477184 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:37.487217 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f3ed1e-6982-4e66-8416-36a895e1cb22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:37.488226 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f3ed1e-6982-4e66-8416-36a895e1cb22 None admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:37.488658 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f3ed1e-6982-4e66-8416-36a895e1cb22 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:37.489080 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f3ed1e-6982-4e66-8416-36a895e1cb22 None admin] PATH_INFO: `/v3/projects/04434e8535a64044996316f1d17dc0bc/users/1564db0771224a05b47d8e3cf778c147/roles/0a9ea6e6d6f94955a5dc949b0746a3f1` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:37.506282 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f3ed1e-6982-4e66-8416-36a895e1cb22 None admin] RBAC: Authorizing `identity:create_grant(project_id=04434e8535a64044996316f1d17dc0bc, user_id=1564db0771224a05b47d8e3cf778c147, role_id=0a9ea6e6d6f94955a5dc949b0746a3f1)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:37.508003 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f3ed1e-6982-4e66-8416-36a895e1cb22 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:37.516333 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1c0611-65b8-4440-b105-9d9243fd63df None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:37.526334 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 298/606] 10.222.0.89 () {58 vars in 1439 bytes} [Thu Jul 28 17:31:37 2022] PUT /identity/v3/projects/04434e8535a64044996316f1d17dc0bc/users/1564db0771224a05b47d8e3cf778c147/roles/0a9ea6e6d6f94955a5dc949b0746a3f1 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:31:37.527980 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f3ed1e-6982-4e66-8416-36a895e1cb22 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:37.558642 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0e3ed5-3cd4-49e7-bdd8-21084692620d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:37.559349 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0e3ed5-3cd4-49e7-bdd8-21084692620d None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:37.559633 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0e3ed5-3cd4-49e7-bdd8-21084692620d None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:37.559906 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0e3ed5-3cd4-49e7-bdd8-21084692620d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:37.568416 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0e3ed5-3cd4-49e7-bdd8-21084692620d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:37.570106 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0e3ed5-3cd4-49e7-bdd8-21084692620d None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:37.585272 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-1f0e3ed5-3cd4-49e7-bdd8-21084692620d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:31:37.585272 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:37.585272 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:37.585272 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:37.585272 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:37.585272 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:37.585272 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:37.585272 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:37.585272 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:37.585272 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:37.585272 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:37.585272 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:37.585272 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 17:31:37.585272 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 17:31:37.585272 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 17:31:37.586237 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 17:31:37.586237 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:37.586237 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:37.586237 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:37.586237 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:37.586237 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:37.586237 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:37.586237 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:37.586237 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:37.586237 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:37.586237 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:37.586237 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:37.586237 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:37.586237 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:37.586237 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:37.586237 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:37.586237 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:37.587558 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 17:31:37.587558 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 17:31:37.587558 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 17:31:37.587558 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 17:31:37.587558 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 17:31:37.587558 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 17:31:37.587558 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:31:37.587558 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:31:37.587558 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 309/607] 10.222.0.89 () {56 vars in 1111 bytes} [Thu Jul 28 17:31:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:31:37.593408 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0e3ed5-3cd4-49e7-bdd8-21084692620d None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:37.666272 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f3ed1e-6982-4e66-8416-36a895e1cb22 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:37.705658 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08f6075-26f5-41d1-a29d-6c7318300f5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:37.706272 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08f6075-26f5-41d1-a29d-6c7318300f5b None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:37.706504 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08f6075-26f5-41d1-a29d-6c7318300f5b None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:37.706781 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08f6075-26f5-41d1-a29d-6c7318300f5b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:37.718868 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08f6075-26f5-41d1-a29d-6c7318300f5b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:31:37.724166 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-a08f6075-26f5-41d1-a29d-6c7318300f5b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 17:31:37.724166 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:37.724166 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:37.724166 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:37.724166 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:37.724166 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:37.724166 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:37.724166 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:37.724166 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:37.724166 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:37.724166 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:37.724166 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:37.724166 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:31:37.724166 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:31:37.724166 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:31:37.724166 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:31:37.725173 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:31:37.725173 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:31:37.725173 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:31:37.725173 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:31:37.725173 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:37.725173 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:37.725173 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:37.725173 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:37.725173 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:37.725173 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:37.725173 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:37.725173 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:37.725173 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:37.725173 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:37.725173 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:37.725173 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:37.726238 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:37.726238 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:37.726238 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:37.726238 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:37.726238 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:31:37.726238 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:31:37.726238 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:31:37.726238 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:31:37.726238 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:31:37.726238 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:31:37.726238 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 17:31:37.726238 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:31:37.726238 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 299/608] 10.222.0.89 () {56 vars in 1102 bytes} [Thu Jul 28 17:31:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:31:37.733906 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08f6075-26f5-41d1-a29d-6c7318300f5b None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:37.742961 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0e3ed5-3cd4-49e7-bdd8-21084692620d None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:37.772223 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:37.789794 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:37.795136 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c02f43-f20b-42ef-a89b-ca249b896db3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:37.795776 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c02f43-f20b-42ef-a89b-ca249b896db3 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:37.796083 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c02f43-f20b-42ef-a89b-ca249b896db3 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:37.796358 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c02f43-f20b-42ef-a89b-ca249b896db3 None admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:37.797485 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c02f43-f20b-42ef-a89b-ca249b896db3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:37.797999 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c02f43-f20b-42ef-a89b-ca249b896db3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:37.799882 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c02f43-f20b-42ef-a89b-ca249b896db3 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:37.818108 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 310/609] 10.222.0.89 () {56 vars in 1148 bytes} [Thu Jul 28 17:31:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:37.828013 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c02f43-f20b-42ef-a89b-ca249b896db3 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:37.896128 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08f6075-26f5-41d1-a29d-6c7318300f5b None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:37.943133 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf016cd-a99e-4feb-880c-9eef93fb63a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:37.943812 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf016cd-a99e-4feb-880c-9eef93fb63a6 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:37.944059 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf016cd-a99e-4feb-880c-9eef93fb63a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:37.944278 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf016cd-a99e-4feb-880c-9eef93fb63a6 None admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:37.944995 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf016cd-a99e-4feb-880c-9eef93fb63a6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:37.945332 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf016cd-a99e-4feb-880c-9eef93fb63a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:37.947215 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf016cd-a99e-4feb-880c-9eef93fb63a6 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:37.958482 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 300/610] 10.222.0.89 () {56 vars in 1103 bytes} [Thu Jul 28 17:31:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:31:37.967364 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c02f43-f20b-42ef-a89b-ca249b896db3 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:37.968669 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf016cd-a99e-4feb-880c-9eef93fb63a6 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:38.039693 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88008f19-f215-4aea-810a-bc79d450da3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:38.042562 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88008f19-f215-4aea-810a-bc79d450da3f None admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:38.045443 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88008f19-f215-4aea-810a-bc79d450da3f None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:38.045937 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88008f19-f215-4aea-810a-bc79d450da3f None admin] PATH_INFO: `/v3/projects/04434e8535a64044996316f1d17dc0bc/users/76817cd541294331b0023069092e8e89/roles/0a9ea6e6d6f94955a5dc949b0746a3f1` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:38.087826 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88008f19-f215-4aea-810a-bc79d450da3f None admin] RBAC: Authorizing `identity:create_grant(project_id=04434e8535a64044996316f1d17dc0bc, user_id=76817cd541294331b0023069092e8e89, role_id=0a9ea6e6d6f94955a5dc949b0746a3f1)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:38.090589 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88008f19-f215-4aea-810a-bc79d450da3f None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:38.117176 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 311/611] 10.222.0.89 () {58 vars in 1439 bytes} [Thu Jul 28 17:31:37 2022] PUT /identity/v3/projects/04434e8535a64044996316f1d17dc0bc/users/76817cd541294331b0023069092e8e89/roles/0a9ea6e6d6f94955a5dc949b0746a3f1 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:31:38.138107 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:38.178284 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf016cd-a99e-4feb-880c-9eef93fb63a6 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:38.224482 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de0d668-6bfe-457a-ba3e-6154087f6e2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:38.225174 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de0d668-6bfe-457a-ba3e-6154087f6e2f None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:38.225466 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de0d668-6bfe-457a-ba3e-6154087f6e2f None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:38.225745 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de0d668-6bfe-457a-ba3e-6154087f6e2f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:38.240473 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de0d668-6bfe-457a-ba3e-6154087f6e2f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:38.242000 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de0d668-6bfe-457a-ba3e-6154087f6e2f None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:38.265685 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-8de0d668-6bfe-457a-ba3e-6154087f6e2f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:31:38.265685 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:38.265685 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:38.265685 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:38.265685 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:38.265685 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:38.265685 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:38.265685 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:38.265685 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:38.265685 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:38.265685 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:38.265685 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:38.265685 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:31:38.265685 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:31:38.265685 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:31:38.266701 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:31:38.266701 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:38.266701 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:38.266701 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:38.266701 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:38.266701 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:38.266701 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:38.266701 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:38.266701 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:38.266701 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:38.266701 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:38.266701 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:38.266701 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:38.266701 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:38.266701 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:38.266701 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:38.266701 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:38.268237 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:31:38.268237 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:31:38.268237 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:31:38.268237 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:31:38.268237 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 301/612] 10.222.0.89 () {56 vars in 1108 bytes} [Thu Jul 28 17:31:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:31:38.274758 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88008f19-f215-4aea-810a-bc79d450da3f None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:38.351381 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:38.422914 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88008f19-f215-4aea-810a-bc79d450da3f None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:38.464161 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989e8f74-8c8e-41c2-8bc5-ea6386f1cf8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:38.464832 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989e8f74-8c8e-41c2-8bc5-ea6386f1cf8d None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:38.465118 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989e8f74-8c8e-41c2-8bc5-ea6386f1cf8d None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:38.465437 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989e8f74-8c8e-41c2-8bc5-ea6386f1cf8d None admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:38.466135 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989e8f74-8c8e-41c2-8bc5-ea6386f1cf8d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:38.466439 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989e8f74-8c8e-41c2-8bc5-ea6386f1cf8d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:38.467963 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989e8f74-8c8e-41c2-8bc5-ea6386f1cf8d None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:38.480420 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 312/613] 10.222.0.89 () {56 vars in 1109 bytes} [Thu Jul 28 17:31:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:38.486181 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de0d668-6bfe-457a-ba3e-6154087f6e2f None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:38.566888 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51f2fd8-d7eb-4142-bdc5-942049c2b0aa None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:38.567240 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51f2fd8-d7eb-4142-bdc5-942049c2b0aa None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:38.567444 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51f2fd8-d7eb-4142-bdc5-942049c2b0aa None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:38.568947 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 313/614] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:31:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 17:31:38.574215 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d90176-5990-4537-978a-c69bd1cc3a04 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:38.574591 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d90176-5990-4537-978a-c69bd1cc3a04 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:38.574808 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d90176-5990-4537-978a-c69bd1cc3a04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:38.610431 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de0d668-6bfe-457a-ba3e-6154087f6e2f None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:38.618335 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-55d90176-5990-4537-978a-c69bd1cc3a04 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:31:38.619612 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-55d90176-5990-4537-978a-c69bd1cc3a04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:31:38.644602 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:38.652547 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8eab973-0137-44a0-8712-dd2a70196acd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:38.653404 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eab973-0137-44a0-8712-dd2a70196acd None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:38.653788 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eab973-0137-44a0-8712-dd2a70196acd None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:38.654795 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eab973-0137-44a0-8712-dd2a70196acd None admin] PATH_INFO: `/v3/users/swift` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:38.663415 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8eab973-0137-44a0-8712-dd2a70196acd None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:38.665056 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8eab973-0137-44a0-8712-dd2a70196acd None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:38.678430 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-d8eab973-0137-44a0-8712-dd2a70196acd None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 28 17:31:38.678430 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:38.678430 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:38.678430 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:38.678430 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:38.678430 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:38.678430 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:38.678430 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:38.678430 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:38.678430 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:38.678430 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:38.678430 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:38.678430 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 17:31:38.678430 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 17:31:38.678430 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 17:31:38.678430 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 17:31:38.679657 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:38.679657 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:38.679657 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 17:31:38.679657 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:31:38.679657 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 17:31:38.679657 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:31:38.679657 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:38.679657 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:38.679657 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:38.679657 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:38.679657 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:38.679657 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:38.679657 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:38.679657 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:38.679657 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:38.679657 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:38.679657 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:38.679657 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:38.680860 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:38.680860 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:38.680860 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 17:31:38.680860 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 17:31:38.680860 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 17:31:38.680860 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 17:31:38.680860 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 17:31:38.680860 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 17:31:38.680860 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 28 17:31:38.680860 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:31:38.683400 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 302/615] 10.222.0.89 () {56 vars in 1096 bytes} [Thu Jul 28 17:31:38 2022] GET /identity/v3/users/swift => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:31:38.691545 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8eab973-0137-44a0-8712-dd2a70196acd None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:38.742807 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 314/616] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:31:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 17:31:38.784831 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d90176-5990-4537-978a-c69bd1cc3a04 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:38.845027 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:38.894148 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8eab973-0137-44a0-8712-dd2a70196acd None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:38.943719 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d90176-5990-4537-978a-c69bd1cc3a04 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:38.965128 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e6d2f5-e410-4ca1-abac-de64d08ebdc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:38.965822 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e6d2f5-e410-4ca1-abac-de64d08ebdc5 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:38.966056 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e6d2f5-e410-4ca1-abac-de64d08ebdc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:38.966331 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e6d2f5-e410-4ca1-abac-de64d08ebdc5 None admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:38.967920 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e6d2f5-e410-4ca1-abac-de64d08ebdc5 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:38.968233 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e6d2f5-e410-4ca1-abac-de64d08ebdc5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:38.969709 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e6d2f5-e410-4ca1-abac-de64d08ebdc5 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:39.006046 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 303/617] 10.222.0.89 () {56 vars in 1135 bytes} [Thu Jul 28 17:31:38 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:39.010696 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e6d2f5-e410-4ca1-abac-de64d08ebdc5 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:39.015662 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a5f31d-f347-44c1-90ad-bd2ad939ec43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:39.016320 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a5f31d-f347-44c1-90ad-bd2ad939ec43 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:39.016601 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a5f31d-f347-44c1-90ad-bd2ad939ec43 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:39.016879 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a5f31d-f347-44c1-90ad-bd2ad939ec43 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:39.043708 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a5f31d-f347-44c1-90ad-bd2ad939ec43 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:31:39.055168 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-a0a5f31d-f347-44c1-90ad-bd2ad939ec43 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 17:31:39.055168 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:39.055168 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:39.055168 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:39.055168 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:39.055168 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:39.055168 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:39.055168 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:39.055168 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:39.055168 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:39.055168 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:39.055168 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:39.055168 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:31:39.055168 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:31:39.055168 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:31:39.055168 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:31:39.056135 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:31:39.056135 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:31:39.056135 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:31:39.056135 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:31:39.056135 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:39.056135 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:39.056135 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:39.056135 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:39.056135 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:39.056135 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:39.056135 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:39.056135 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:39.056135 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:39.056135 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:39.056135 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:39.056135 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:39.057441 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:39.057441 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:39.057441 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:39.057441 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:39.057441 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:31:39.057441 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:31:39.057441 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:31:39.057441 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:31:39.057441 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:31:39.057441 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:31:39.057441 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 17:31:39.057441 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:31:39.057441 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 315/618] 10.222.0.89 () {56 vars in 1102 bytes} [Thu Jul 28 17:31:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:31:39.062914 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a5f31d-f347-44c1-90ad-bd2ad939ec43 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:39.104193 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:39.164992 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e6d2f5-e410-4ca1-abac-de64d08ebdc5 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:39.210862 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f0b147-f8e8-4ae5-92f5-97f4fdecb59c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:39.211525 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f0b147-f8e8-4ae5-92f5-97f4fdecb59c None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:39.211806 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f0b147-f8e8-4ae5-92f5-97f4fdecb59c None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:39.212079 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f0b147-f8e8-4ae5-92f5-97f4fdecb59c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:39.230574 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f0b147-f8e8-4ae5-92f5-97f4fdecb59c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:39.234741 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f0b147-f8e8-4ae5-92f5-97f4fdecb59c None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:39.253004 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-01f0b147-f8e8-4ae5-92f5-97f4fdecb59c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:31:39.253004 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:39.253004 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:39.253004 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:39.253004 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:39.253004 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:39.253004 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:39.253004 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:39.253004 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:39.253004 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:39.253004 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:39.253004 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:39.253004 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:31:39.253004 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:31:39.253004 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:31:39.254172 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:31:39.254172 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:39.254172 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:39.254172 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:39.254172 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:39.254172 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:39.254172 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:39.254172 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:39.254172 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:39.254172 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:39.254172 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:39.254172 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:39.254172 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:39.254172 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:39.254172 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:39.254172 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:39.254172 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:39.255099 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:31:39.255099 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:31:39.255099 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:31:39.255099 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:31:39.257914 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 304/619] 10.222.0.89 () {56 vars in 1108 bytes} [Thu Jul 28 17:31:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:31:39.263238 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f0b147-f8e8-4ae5-92f5-97f4fdecb59c None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:39.268438 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:39.286404 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a5f31d-f347-44c1-90ad-bd2ad939ec43 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:39.355359 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbfcbf4-56f1-46bd-8c1b-9ccdc75c00da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:39.356377 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbfcbf4-56f1-46bd-8c1b-9ccdc75c00da None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:39.358257 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbfcbf4-56f1-46bd-8c1b-9ccdc75c00da None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:39.358897 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbfcbf4-56f1-46bd-8c1b-9ccdc75c00da None admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:39.360953 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbfcbf4-56f1-46bd-8c1b-9ccdc75c00da None admin] RBAC: Adding query filter params (name=service) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:39.361538 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbfcbf4-56f1-46bd-8c1b-9ccdc75c00da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:39.363875 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbfcbf4-56f1-46bd-8c1b-9ccdc75c00da None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:39.381660 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 316/620] 10.222.0.89 () {56 vars in 1103 bytes} [Thu Jul 28 17:31:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:39.384721 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbfcbf4-56f1-46bd-8c1b-9ccdc75c00da None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:39.453526 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:39.464996 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f0b147-f8e8-4ae5-92f5-97f4fdecb59c None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:39.528568 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db588785-907c-4b0d-bd7f-8eebefdea056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:39.529331 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db588785-907c-4b0d-bd7f-8eebefdea056 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:39.529768 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db588785-907c-4b0d-bd7f-8eebefdea056 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:39.529768 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db588785-907c-4b0d-bd7f-8eebefdea056 None admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:39.533056 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db588785-907c-4b0d-bd7f-8eebefdea056 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:39.533520 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db588785-907c-4b0d-bd7f-8eebefdea056 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:39.535012 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db588785-907c-4b0d-bd7f-8eebefdea056 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:39.562368 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 305/621] 10.222.0.89 () {56 vars in 1109 bytes} [Thu Jul 28 17:31:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:39.568077 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db588785-907c-4b0d-bd7f-8eebefdea056 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:39.583359 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbfcbf4-56f1-46bd-8c1b-9ccdc75c00da None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:39.619369 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f72085-7dca-4615-82ce-c7eb0955aacd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:39.620422 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f72085-7dca-4615-82ce-c7eb0955aacd None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:39.620799 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f72085-7dca-4615-82ce-c7eb0955aacd None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:39.621183 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f72085-7dca-4615-82ce-c7eb0955aacd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:39.633333 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f72085-7dca-4615-82ce-c7eb0955aacd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:39.635018 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f72085-7dca-4615-82ce-c7eb0955aacd None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:39.650971 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-33f72085-7dca-4615-82ce-c7eb0955aacd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:31:39.650971 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:39.650971 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:39.650971 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:39.650971 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:39.650971 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:39.650971 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:39.650971 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:39.650971 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:39.650971 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:39.650971 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:39.650971 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:39.650971 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:31:39.650971 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:31:39.650971 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:31:39.652105 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:31:39.652105 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:39.652105 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:39.652105 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:39.652105 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:39.652105 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:39.652105 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:39.652105 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:39.652105 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:39.652105 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:39.652105 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:39.652105 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:39.652105 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:39.652105 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:39.652105 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:39.652105 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:39.652105 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:39.653228 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:31:39.653228 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:31:39.653228 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:31:39.653228 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:31:39.654948 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 317/622] 10.222.0.89 () {56 vars in 1108 bytes} [Thu Jul 28 17:31:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:31:39.656476 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519325d6-4f38-474e-ac47-43a5fec3e376 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:39.656878 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519325d6-4f38-474e-ac47-43a5fec3e376 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:39.657276 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519325d6-4f38-474e-ac47-43a5fec3e376 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:39.658952 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 318/623] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:31:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:31:39.663885 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519325d6-4f38-474e-ac47-43a5fec3e376 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:39.667847 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:39.687365 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db588785-907c-4b0d-bd7f-8eebefdea056 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:39.725222 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28a3d78-48ff-4ccb-9f55-9bc9853f8fef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:39.725877 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28a3d78-48ff-4ccb-9f55-9bc9853f8fef None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:39.726159 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28a3d78-48ff-4ccb-9f55-9bc9853f8fef None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:39.726433 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28a3d78-48ff-4ccb-9f55-9bc9853f8fef None admin] PATH_INFO: `/v3/projects/service` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:39.734093 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28a3d78-48ff-4ccb-9f55-9bc9853f8fef None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:39.735548 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28a3d78-48ff-4ccb-9f55-9bc9853f8fef None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:39.747560 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-e28a3d78-48ff-4ccb-9f55-9bc9853f8fef None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:31:39.747560 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:39.747560 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:39.747560 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:39.747560 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:39.747560 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:39.747560 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:39.747560 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:39.747560 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:39.747560 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:39.747560 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:39.747560 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:39.747560 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 17:31:39.747560 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 17:31:39.747560 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 17:31:39.749758 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 17:31:39.749758 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:39.749758 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:39.749758 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:39.749758 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:39.749758 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:39.749758 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:39.749758 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:39.749758 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:39.749758 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:39.749758 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:39.749758 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:39.749758 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:39.749758 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:39.749758 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:39.749758 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:39.749758 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:39.750645 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 17:31:39.750645 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 17:31:39.750645 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 17:31:39.750645 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 17:31:39.750645 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 17:31:39.750645 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 17:31:39.750645 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:31:39.750645 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:31:39.750645 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 306/624] 10.222.0.89 () {56 vars in 1111 bytes} [Thu Jul 28 17:31:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:31:39.751144 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b767da-7ca1-4b33-8070-3cd78acf20e9 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:39.751386 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b767da-7ca1-4b33-8070-3cd78acf20e9 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:39.751818 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b767da-7ca1-4b33-8070-3cd78acf20e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:39.769986 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-d5b767da-7ca1-4b33-8070-3cd78acf20e9 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:31:39.771213 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-d5b767da-7ca1-4b33-8070-3cd78acf20e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:31:39.780243 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519325d6-4f38-474e-ac47-43a5fec3e376 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:39.789575 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:39.829490 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04ffd51-9e2e-4937-8176-f94d28bc68f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:39.830389 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04ffd51-9e2e-4937-8176-f94d28bc68f8 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:39.830873 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04ffd51-9e2e-4937-8176-f94d28bc68f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:39.831343 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04ffd51-9e2e-4937-8176-f94d28bc68f8 None admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:39.832270 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04ffd51-9e2e-4937-8176-f94d28bc68f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:39.832806 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04ffd51-9e2e-4937-8176-f94d28bc68f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:39.834440 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04ffd51-9e2e-4937-8176-f94d28bc68f8 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:39.848921 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 319/625] 10.222.0.89 () {56 vars in 1109 bytes} [Thu Jul 28 17:31:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:39.857202 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 307/626] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:31:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:31:39.857202 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b767da-7ca1-4b33-8070-3cd78acf20e9 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:39.858317 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04ffd51-9e2e-4937-8176-f94d28bc68f8 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:39.941549 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:39.959811 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b767da-7ca1-4b33-8070-3cd78acf20e9 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:39.983332 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04ffd51-9e2e-4937-8176-f94d28bc68f8 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:39.995231 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c916753-3680-46ad-9068-ff65e6712e80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:39.995886 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c916753-3680-46ad-9068-ff65e6712e80 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:39.996170 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c916753-3680-46ad-9068-ff65e6712e80 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:39.996445 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c916753-3680-46ad-9068-ff65e6712e80 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:40.005089 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c916753-3680-46ad-9068-ff65e6712e80 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:40.006610 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c916753-3680-46ad-9068-ff65e6712e80 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:40.019497 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-9c916753-3680-46ad-9068-ff65e6712e80 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 17:31:40.019497 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:40.019497 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:40.019497 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:40.019497 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:40.019497 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:40.019497 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:40.019497 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:40.019497 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:40.019497 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:40.019497 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:40.019497 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:40.019497 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 17:31:40.019497 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 17:31:40.019497 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 17:31:40.019497 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 17:31:40.020520 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:40.020520 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:40.020520 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 17:31:40.020520 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:31:40.020520 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 17:31:40.020520 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:31:40.020520 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:40.020520 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:40.020520 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:40.020520 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:40.020520 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:40.020520 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:40.020520 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:40.020520 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:40.020520 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:40.020520 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:40.020520 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:40.020520 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:40.022094 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:40.022094 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:40.022094 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 17:31:40.022094 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 17:31:40.022094 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 17:31:40.022094 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 17:31:40.022094 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 17:31:40.022094 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 17:31:40.022094 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 17:31:40.022094 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:31:40.022094 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 308/627] 10.222.0.89 () {56 vars in 1093 bytes} [Thu Jul 28 17:31:39 2022] GET /identity/v3/users/nova => generated 82 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:31:40.022094 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c916753-3680-46ad-9068-ff65e6712e80 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:40.027845 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93a42d6-068d-4c9b-816d-70a0061518e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:40.028743 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93a42d6-068d-4c9b-816d-70a0061518e2 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:40.029838 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93a42d6-068d-4c9b-816d-70a0061518e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:40.030372 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93a42d6-068d-4c9b-816d-70a0061518e2 None admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:40.031446 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93a42d6-068d-4c9b-816d-70a0061518e2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:40.031970 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93a42d6-068d-4c9b-816d-70a0061518e2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:40.033740 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93a42d6-068d-4c9b-816d-70a0061518e2 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:40.037527 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:40.047101 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 320/628] 10.222.0.89 () {56 vars in 1148 bytes} [Thu Jul 28 17:31:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:40.048288 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93a42d6-068d-4c9b-816d-70a0061518e2 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:40.148194 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c916753-3680-46ad-9068-ff65e6712e80 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:40.163154 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93a42d6-068d-4c9b-816d-70a0061518e2 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:40.189461 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:40.201896 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61dd6f8-6386-457d-afa5-2cec86a00f14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:40.202253 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b963340b-b732-4c20-9f39-c162e969a33b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:40.202801 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61dd6f8-6386-457d-afa5-2cec86a00f14 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:40.202993 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b963340b-b732-4c20-9f39-c162e969a33b None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:40.203155 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61dd6f8-6386-457d-afa5-2cec86a00f14 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:40.203296 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b963340b-b732-4c20-9f39-c162e969a33b None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:40.203425 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61dd6f8-6386-457d-afa5-2cec86a00f14 None admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:40.203505 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b963340b-b732-4c20-9f39-c162e969a33b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:40.204124 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61dd6f8-6386-457d-afa5-2cec86a00f14 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:40.204433 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61dd6f8-6386-457d-afa5-2cec86a00f14 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:40.205888 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61dd6f8-6386-457d-afa5-2cec86a00f14 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:40.214353 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b963340b-b732-4c20-9f39-c162e969a33b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:31:40.220729 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-b963340b-b732-4c20-9f39-c162e969a33b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 17:31:40.220729 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:40.220729 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:40.220729 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:40.220729 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:40.220729 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:40.220729 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:40.220729 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:40.220729 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:40.220729 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:40.220729 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:40.220729 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:40.220729 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:31:40.220729 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:31:40.220729 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:31:40.220729 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:31:40.221979 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:31:40.221979 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:31:40.221979 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:31:40.221979 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:31:40.221979 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:40.221979 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:40.221979 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:40.221979 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:40.221979 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:40.221979 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:40.221979 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:40.221979 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:40.221979 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:40.221979 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:40.221979 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:40.221979 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:40.222807 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:40.222807 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:40.222807 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:40.222807 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:40.222807 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:31:40.222807 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:31:40.222807 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:31:40.222807 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:31:40.222807 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:31:40.222807 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:31:40.222807 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 17:31:40.222807 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:31:40.222807 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 309/629] 10.222.0.89 () {56 vars in 1102 bytes} [Thu Jul 28 17:31:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:31:40.223540 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b963340b-b732-4c20-9f39-c162e969a33b None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:40.228534 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 321/630] 10.222.0.89 () {56 vars in 1133 bytes} [Thu Jul 28 17:31:40 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:40.229986 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef09279a-15c0-4b82-908d-e5d1ec39f8a3 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:40.231009 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef09279a-15c0-4b82-908d-e5d1ec39f8a3 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:40.231009 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef09279a-15c0-4b82-908d-e5d1ec39f8a3 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:40.237187 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 322/631] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:31:40 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:31:40.238223 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef09279a-15c0-4b82-908d-e5d1ec39f8a3 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:40.300113 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:40.339091 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b963340b-b732-4c20-9f39-c162e969a33b None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:40.347677 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef09279a-15c0-4b82-908d-e5d1ec39f8a3 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:40.375727 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73516c2e-4e56-487d-85a1-f8735435bc04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:40.376411 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73516c2e-4e56-487d-85a1-f8735435bc04 None admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:40.376691 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73516c2e-4e56-487d-85a1-f8735435bc04 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:40.376967 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73516c2e-4e56-487d-85a1-f8735435bc04 None admin] PATH_INFO: `/v3/projects/04434e8535a64044996316f1d17dc0bc/users/3c59473b01bf4433ba3b111114fc048b/roles/0a9ea6e6d6f94955a5dc949b0746a3f1` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:40.387187 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25847e42-c60f-414f-b0fa-3fd111c886aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:40.387922 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25847e42-c60f-414f-b0fa-3fd111c886aa None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:40.388305 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25847e42-c60f-414f-b0fa-3fd111c886aa None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:40.388685 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25847e42-c60f-414f-b0fa-3fd111c886aa None admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:40.389547 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25847e42-c60f-414f-b0fa-3fd111c886aa None admin] RBAC: Adding query filter params (name=service) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:40.390153 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25847e42-c60f-414f-b0fa-3fd111c886aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:40.391316 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25847e42-c60f-414f-b0fa-3fd111c886aa None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:40.392932 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73516c2e-4e56-487d-85a1-f8735435bc04 None admin] RBAC: Authorizing `identity:create_grant(project_id=04434e8535a64044996316f1d17dc0bc, user_id=3c59473b01bf4433ba3b111114fc048b, role_id=0a9ea6e6d6f94955a5dc949b0746a3f1)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:40.394404 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73516c2e-4e56-487d-85a1-f8735435bc04 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:40.400255 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 323/632] 10.222.0.89 () {56 vars in 1103 bytes} [Thu Jul 28 17:31:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:40.401213 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25847e42-c60f-414f-b0fa-3fd111c886aa None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:40.417462 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 310/633] 10.222.0.89 () {58 vars in 1439 bytes} [Thu Jul 28 17:31:40 2022] PUT /identity/v3/projects/04434e8535a64044996316f1d17dc0bc/users/3c59473b01bf4433ba3b111114fc048b/roles/0a9ea6e6d6f94955a5dc949b0746a3f1 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:31:40.430734 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24340fef-5050-4905-a949-455aa0e2f881 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:40.431025 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24340fef-5050-4905-a949-455aa0e2f881 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:40.431326 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24340fef-5050-4905-a949-455aa0e2f881 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:40.453233 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-24340fef-5050-4905-a949-455aa0e2f881 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:31:40.454634 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-24340fef-5050-4905-a949-455aa0e2f881 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:31:40.465182 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:40.551108 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25847e42-c60f-414f-b0fa-3fd111c886aa None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:40.553329 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 311/634] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:31:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:31:40.554476 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24340fef-5050-4905-a949-455aa0e2f881 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:40.607896 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b5edd5-87f8-4e67-a488-3e4c40796220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:40.608660 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b5edd5-87f8-4e67-a488-3e4c40796220 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:40.608955 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b5edd5-87f8-4e67-a488-3e4c40796220 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:40.609286 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b5edd5-87f8-4e67-a488-3e4c40796220 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:40.615218 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:40.674113 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24340fef-5050-4905-a949-455aa0e2f881 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:40.689331 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b5edd5-87f8-4e67-a488-3e4c40796220 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:40.691112 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b5edd5-87f8-4e67-a488-3e4c40796220 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:40.715799 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62869ef0-6361-4589-a60e-288ad883c1df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:40.716485 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62869ef0-6361-4589-a60e-288ad883c1df None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:40.716860 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62869ef0-6361-4589-a60e-288ad883c1df None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:40.717301 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62869ef0-6361-4589-a60e-288ad883c1df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:40.729451 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62869ef0-6361-4589-a60e-288ad883c1df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:40.731000 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62869ef0-6361-4589-a60e-288ad883c1df None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:40.740869 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:40.747783 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-62869ef0-6361-4589-a60e-288ad883c1df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:31:40.747783 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:40.747783 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:40.747783 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:40.747783 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:40.747783 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:40.747783 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:40.747783 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:40.747783 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:40.747783 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:40.747783 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:40.747783 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:40.747783 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:31:40.747783 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:31:40.747783 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:31:40.749620 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:31:40.749620 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:40.749620 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:40.749620 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:40.749620 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:40.749620 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:40.749620 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:40.749620 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:40.749620 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:40.749620 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:40.749620 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:40.749620 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:40.749620 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:40.749620 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:40.749620 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:40.749620 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:40.749620 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:40.750547 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:31:40.750547 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:31:40.750547 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:31:40.750547 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:31:40.754325 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 312/635] 10.222.0.89 () {56 vars in 1108 bytes} [Thu Jul 28 17:31:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:31:40.755136 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62869ef0-6361-4589-a60e-288ad883c1df None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:40.804992 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 324/636] 10.222.0.89 () {58 vars in 1282 bytes} [Thu Jul 28 17:31:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 17:31:40.806228 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c3cc35-34e8-43fc-9160-1ccd2db9b597 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:40.806513 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c3cc35-34e8-43fc-9160-1ccd2db9b597 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:40.806792 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c3cc35-34e8-43fc-9160-1ccd2db9b597 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:40.812087 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 325/637] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:31:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:31:40.812087 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c3cc35-34e8-43fc-9160-1ccd2db9b597 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:40.876030 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:40.880041 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62869ef0-6361-4589-a60e-288ad883c1df None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:40.917131 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401d83d8-3ca3-4ebc-ac7c-eba273894267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:40.917791 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401d83d8-3ca3-4ebc-ac7c-eba273894267 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:40.918067 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401d83d8-3ca3-4ebc-ac7c-eba273894267 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:40.918330 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401d83d8-3ca3-4ebc-ac7c-eba273894267 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:40.927834 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c3cc35-34e8-43fc-9160-1ccd2db9b597 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:40.928906 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401d83d8-3ca3-4ebc-ac7c-eba273894267 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:31:40.935277 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-401d83d8-3ca3-4ebc-ac7c-eba273894267 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 17:31:40.935277 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:40.935277 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:40.935277 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:40.935277 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:40.935277 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:40.935277 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:40.935277 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:40.935277 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:40.935277 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:40.935277 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:40.935277 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:40.935277 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:31:40.935277 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:31:40.935277 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:31:40.935277 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:31:40.936942 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:31:40.936942 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:31:40.936942 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:31:40.936942 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:31:40.936942 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:40.936942 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:40.936942 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:40.936942 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:40.936942 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:40.936942 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:40.936942 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:40.936942 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:40.936942 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:40.936942 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:40.936942 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:40.936942 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:40.937851 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:40.937851 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:40.937851 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:40.937851 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:40.937851 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:31:40.937851 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:31:40.937851 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:31:40.937851 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:31:40.937851 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:31:40.937851 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:31:40.937851 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 17:31:40.937851 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:31:40.937851 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 313/638] 10.222.0.89 () {56 vars in 1102 bytes} [Thu Jul 28 17:31:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:31:40.938557 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401d83d8-3ca3-4ebc-ac7c-eba273894267 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:40.970260 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61aee1f8-d3b4-4704-8beb-f047a2877bf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:40.970910 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aee1f8-d3b4-4704-8beb-f047a2877bf3 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:40.971208 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aee1f8-d3b4-4704-8beb-f047a2877bf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:40.971507 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aee1f8-d3b4-4704-8beb-f047a2877bf3 None admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:40.972242 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61aee1f8-d3b4-4704-8beb-f047a2877bf3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:40.972551 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61aee1f8-d3b4-4704-8beb-f047a2877bf3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:40.974070 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61aee1f8-d3b4-4704-8beb-f047a2877bf3 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:40.986998 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 326/639] 10.222.0.89 () {56 vars in 1109 bytes} [Thu Jul 28 17:31:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:40.989191 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f96069-7049-4be9-92f4-385088470263 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:40.989848 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f96069-7049-4be9-92f4-385088470263 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:40.990417 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f96069-7049-4be9-92f4-385088470263 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:41.000694 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:41.013419 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-c0f96069-7049-4be9-92f4-385088470263 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:31:41.014876 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-c0f96069-7049-4be9-92f4-385088470263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:31:41.056667 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401d83d8-3ca3-4ebc-ac7c-eba273894267 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:41.093202 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 327/640] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:31:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:31:41.093202 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f96069-7049-4be9-92f4-385088470263 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:41.098539 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15bfb73-6a9d-45c0-84cd-e451664ab490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:41.099252 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15bfb73-6a9d-45c0-84cd-e451664ab490 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:41.099602 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15bfb73-6a9d-45c0-84cd-e451664ab490 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:41.099946 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15bfb73-6a9d-45c0-84cd-e451664ab490 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:41.113075 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15bfb73-6a9d-45c0-84cd-e451664ab490 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:41.116054 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15bfb73-6a9d-45c0-84cd-e451664ab490 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:41.133603 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-f15bfb73-6a9d-45c0-84cd-e451664ab490 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:31:41.133603 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:41.133603 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:41.133603 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:41.133603 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:41.133603 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:41.133603 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:41.133603 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:41.133603 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:41.133603 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:41.133603 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:41.133603 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:41.133603 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:31:41.133603 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:31:41.133603 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:31:41.134600 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:31:41.134600 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:41.134600 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:41.134600 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:41.134600 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:41.134600 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:41.134600 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:41.134600 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:41.134600 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:41.134600 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:41.134600 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:41.134600 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:41.134600 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:41.134600 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:41.134600 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:41.134600 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:41.134600 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:41.135500 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:31:41.135500 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:31:41.135500 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:31:41.135500 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:31:41.135500 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 314/641] 10.222.0.89 () {56 vars in 1108 bytes} [Thu Jul 28 17:31:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:31:41.135788 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15bfb73-6a9d-45c0-84cd-e451664ab490 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:41.151832 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:41.212210 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f96069-7049-4be9-92f4-385088470263 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:41.249148 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04029711-5e62-40c5-8994-d1ddc4312903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:41.249787 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04029711-5e62-40c5-8994-d1ddc4312903 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:41.250055 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04029711-5e62-40c5-8994-d1ddc4312903 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:41.250327 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04029711-5e62-40c5-8994-d1ddc4312903 None admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:41.251032 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04029711-5e62-40c5-8994-d1ddc4312903 None admin] RBAC: Adding query filter params (name=service) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:41.251354 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04029711-5e62-40c5-8994-d1ddc4312903 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:41.251726 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15bfb73-6a9d-45c0-84cd-e451664ab490 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:41.252707 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04029711-5e62-40c5-8994-d1ddc4312903 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:41.254090 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:41.262592 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 328/642] 10.222.0.89 () {56 vars in 1103 bytes} [Thu Jul 28 17:31:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:41.263462 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04029711-5e62-40c5-8994-d1ddc4312903 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:41.292665 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0add442-7663-4876-95d2-ed962e39c26c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:41.293350 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0add442-7663-4876-95d2-ed962e39c26c None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:41.293632 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0add442-7663-4876-95d2-ed962e39c26c None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:41.293905 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0add442-7663-4876-95d2-ed962e39c26c None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:41.303617 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0add442-7663-4876-95d2-ed962e39c26c None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:41.305095 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0add442-7663-4876-95d2-ed962e39c26c None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:41.318578 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-a0add442-7663-4876-95d2-ed962e39c26c None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 28 17:31:41.318578 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:41.318578 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:41.318578 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:41.318578 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:41.318578 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:41.318578 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:41.318578 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:41.318578 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:41.318578 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:41.318578 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:41.318578 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:41.318578 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 17:31:41.318578 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 17:31:41.318578 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 17:31:41.318578 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 17:31:41.319776 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:41.319776 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:41.319776 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 17:31:41.319776 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:31:41.319776 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 17:31:41.319776 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:31:41.319776 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:41.319776 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:41.319776 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:41.319776 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:41.319776 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:41.319776 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:41.319776 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:41.319776 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:41.319776 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:41.319776 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:41.319776 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:41.319776 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:41.320751 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:41.320751 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:41.320751 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 17:31:41.320751 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 17:31:41.320751 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 17:31:41.320751 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 17:31:41.320751 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 17:31:41.320751 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 17:31:41.320751 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 28 17:31:41.320751 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:31:41.320751 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 315/643] 10.222.0.89 () {56 vars in 1099 bytes} [Thu Jul 28 17:31:41 2022] GET /identity/v3/users/cinder => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:31:41.320751 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0add442-7663-4876-95d2-ed962e39c26c None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:41.374613 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04029711-5e62-40c5-8994-d1ddc4312903 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:41.396060 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:41.413848 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c715cfe-eaa9-4c8b-9efc-a7863387eb32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:41.414617 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c715cfe-eaa9-4c8b-9efc-a7863387eb32 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:41.414903 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c715cfe-eaa9-4c8b-9efc-a7863387eb32 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:41.415177 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c715cfe-eaa9-4c8b-9efc-a7863387eb32 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:41.426547 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c715cfe-eaa9-4c8b-9efc-a7863387eb32 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:31:41.429572 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0add442-7663-4876-95d2-ed962e39c26c None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:41.432218 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-7c715cfe-eaa9-4c8b-9efc-a7863387eb32 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 17:31:41.432218 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:41.432218 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:41.432218 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:41.432218 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:41.432218 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:41.432218 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:41.432218 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:41.432218 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:41.432218 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:41.432218 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:41.432218 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:41.432218 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:31:41.432218 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:31:41.432218 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:31:41.432218 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:31:41.433152 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:31:41.433152 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:31:41.433152 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:31:41.433152 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:31:41.433152 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:41.433152 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:41.433152 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:41.433152 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:41.433152 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:41.433152 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:41.433152 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:41.433152 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:41.433152 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:41.433152 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:41.433152 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:41.433152 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:41.434244 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:41.434244 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:41.434244 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:41.434244 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:41.434244 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:31:41.434244 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:31:41.434244 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:31:41.434244 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:31:41.434244 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:31:41.434244 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:31:41.434244 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 17:31:41.434244 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:31:41.434244 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 329/644] 10.222.0.89 () {56 vars in 1102 bytes} [Thu Jul 28 17:31:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:31:41.434244 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c715cfe-eaa9-4c8b-9efc-a7863387eb32 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:41.473601 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e06a9ea-4bc7-456a-9b09-ab75f9bd0aa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:41.474230 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e06a9ea-4bc7-456a-9b09-ab75f9bd0aa0 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:41.474510 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e06a9ea-4bc7-456a-9b09-ab75f9bd0aa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:41.474782 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e06a9ea-4bc7-456a-9b09-ab75f9bd0aa0 None admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:41.475475 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e06a9ea-4bc7-456a-9b09-ab75f9bd0aa0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:41.475780 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e06a9ea-4bc7-456a-9b09-ab75f9bd0aa0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:41.477141 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e06a9ea-4bc7-456a-9b09-ab75f9bd0aa0 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:41.489940 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 316/645] 10.222.0.89 () {56 vars in 1109 bytes} [Thu Jul 28 17:31:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:41.495457 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e06a9ea-4bc7-456a-9b09-ab75f9bd0aa0 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:41.500568 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:41.550451 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c715cfe-eaa9-4c8b-9efc-a7863387eb32 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:41.588757 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061bcb4c-df6f-43d9-a1f3-6b601d485ddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:41.589636 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061bcb4c-df6f-43d9-a1f3-6b601d485ddf None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:41.589636 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061bcb4c-df6f-43d9-a1f3-6b601d485ddf None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:41.589993 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061bcb4c-df6f-43d9-a1f3-6b601d485ddf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:41.601342 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061bcb4c-df6f-43d9-a1f3-6b601d485ddf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:41.602786 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061bcb4c-df6f-43d9-a1f3-6b601d485ddf None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:41.609288 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e06a9ea-4bc7-456a-9b09-ab75f9bd0aa0 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:41.618425 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-061bcb4c-df6f-43d9-a1f3-6b601d485ddf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:31:41.618425 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:41.618425 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:41.618425 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:41.618425 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:41.618425 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:41.618425 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:41.618425 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:41.618425 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:41.618425 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:41.618425 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:41.618425 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:41.618425 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:31:41.618425 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:31:41.618425 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:31:41.619316 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:31:41.619316 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:41.619316 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:41.619316 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:41.619316 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:41.619316 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:41.619316 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:41.619316 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:41.619316 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:41.619316 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:41.619316 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:41.619316 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:41.619316 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:41.619316 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:41.619316 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:41.619316 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:41.619316 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:41.620466 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:31:41.620466 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:31:41.620466 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:31:41.620466 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:31:41.620466 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 330/646] 10.222.0.89 () {56 vars in 1108 bytes} [Thu Jul 28 17:31:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:31:41.620745 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061bcb4c-df6f-43d9-a1f3-6b601d485ddf None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:41.645874 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:41.655153 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478ad4be-50c9-4216-abdf-e1cc28a5641d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:41.655930 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478ad4be-50c9-4216-abdf-e1cc28a5641d None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:41.656333 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478ad4be-50c9-4216-abdf-e1cc28a5641d None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:41.656612 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478ad4be-50c9-4216-abdf-e1cc28a5641d None admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:41.657578 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478ad4be-50c9-4216-abdf-e1cc28a5641d None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:41.658191 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478ad4be-50c9-4216-abdf-e1cc28a5641d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:41.659874 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478ad4be-50c9-4216-abdf-e1cc28a5641d None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:41.680018 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 317/647] 10.222.0.89 () {56 vars in 1137 bytes} [Thu Jul 28 17:31:41 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:41.680820 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478ad4be-50c9-4216-abdf-e1cc28a5641d None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:41.740029 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061bcb4c-df6f-43d9-a1f3-6b601d485ddf None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:41.750267 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:41.781543 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db45ad3-a130-4549-aa72-cb1fa17d4535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:41.782169 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db45ad3-a130-4549-aa72-cb1fa17d4535 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:41.782448 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db45ad3-a130-4549-aa72-cb1fa17d4535 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:41.782726 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db45ad3-a130-4549-aa72-cb1fa17d4535 None admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:41.783485 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db45ad3-a130-4549-aa72-cb1fa17d4535 None admin] RBAC: Adding query filter params (name=service) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:41.783742 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db45ad3-a130-4549-aa72-cb1fa17d4535 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:41.785205 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db45ad3-a130-4549-aa72-cb1fa17d4535 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:41.788712 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478ad4be-50c9-4216-abdf-e1cc28a5641d None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:41.798526 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 331/648] 10.222.0.89 () {56 vars in 1103 bytes} [Thu Jul 28 17:31:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:41.798526 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db45ad3-a130-4549-aa72-cb1fa17d4535 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:41.837538 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a7fb3d-a7e7-4e4c-bd7d-8369dae98516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:41.838464 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a7fb3d-a7e7-4e4c-bd7d-8369dae98516 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:41.839040 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a7fb3d-a7e7-4e4c-bd7d-8369dae98516 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:41.839516 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a7fb3d-a7e7-4e4c-bd7d-8369dae98516 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:41.848873 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a7fb3d-a7e7-4e4c-bd7d-8369dae98516 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:41.850649 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a7fb3d-a7e7-4e4c-bd7d-8369dae98516 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:41.866591 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-20a7fb3d-a7e7-4e4c-bd7d-8369dae98516 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:31:41.866591 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:41.866591 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:41.866591 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:41.866591 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:41.866591 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:41.866591 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:41.866591 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:41.866591 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:41.866591 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:41.866591 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:41.866591 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:41.866591 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 17:31:41.866591 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 17:31:41.866591 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 17:31:41.867525 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 17:31:41.867525 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:41.867525 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:41.867525 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:41.867525 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:41.867525 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:41.867525 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:41.867525 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:41.867525 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:41.867525 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:41.867525 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:41.867525 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:41.867525 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:41.867525 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:41.867525 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:41.867525 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:41.867525 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:41.868436 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 17:31:41.868436 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 17:31:41.868436 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 17:31:41.868436 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 17:31:41.868436 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 17:31:41.868436 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 17:31:41.868436 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:31:41.868436 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:31:41.868436 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 318/649] 10.222.0.89 () {56 vars in 1111 bytes} [Thu Jul 28 17:31:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:31:41.868953 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a7fb3d-a7e7-4e4c-bd7d-8369dae98516 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:41.916556 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db45ad3-a130-4549-aa72-cb1fa17d4535 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:41.924285 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:41.955354 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7814235d-9595-4007-a371-042ca986629c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:41.956034 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7814235d-9595-4007-a371-042ca986629c None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:41.956248 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7814235d-9595-4007-a371-042ca986629c None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:41.956517 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7814235d-9595-4007-a371-042ca986629c None admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:41.957743 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7814235d-9595-4007-a371-042ca986629c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:41.958051 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7814235d-9595-4007-a371-042ca986629c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:41.959778 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7814235d-9595-4007-a371-042ca986629c None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:41.977879 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 332/650] 10.222.0.89 () {56 vars in 1109 bytes} [Thu Jul 28 17:31:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:41.977879 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7814235d-9595-4007-a371-042ca986629c None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:41.989683 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a7fb3d-a7e7-4e4c-bd7d-8369dae98516 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:42.012842 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:42.030132 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba6c13a-6507-4a78-8bf9-1dbcdc68a0c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:42.030782 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba6c13a-6507-4a78-8bf9-1dbcdc68a0c4 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:42.031058 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba6c13a-6507-4a78-8bf9-1dbcdc68a0c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:42.031326 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba6c13a-6507-4a78-8bf9-1dbcdc68a0c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:42.102635 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba6c13a-6507-4a78-8bf9-1dbcdc68a0c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:42.104028 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba6c13a-6507-4a78-8bf9-1dbcdc68a0c4 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:42.105365 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7814235d-9595-4007-a371-042ca986629c None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:42.144580 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22064cd4-79b6-4404-8763-4e1719164104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:42.145246 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22064cd4-79b6-4404-8763-4e1719164104 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:42.145524 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22064cd4-79b6-4404-8763-4e1719164104 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:42.145794 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22064cd4-79b6-4404-8763-4e1719164104 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:42.158230 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22064cd4-79b6-4404-8763-4e1719164104 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:42.159709 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22064cd4-79b6-4404-8763-4e1719164104 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:42.161293 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:42.177765 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-22064cd4-79b6-4404-8763-4e1719164104 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:31:42.177765 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:42.177765 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:42.177765 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:42.177765 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:42.177765 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:42.177765 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:42.177765 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:42.177765 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:42.177765 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:42.177765 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:42.177765 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:42.177765 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:31:42.177765 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:31:42.177765 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:31:42.178957 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:31:42.178957 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:42.178957 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:42.178957 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:42.178957 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:42.178957 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:42.178957 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:42.178957 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:42.178957 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:42.178957 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:42.178957 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:42.178957 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:42.178957 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:42.178957 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:42.178957 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:42.178957 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:42.178957 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:42.179987 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:31:42.179987 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:31:42.179987 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:31:42.179987 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:31:42.179987 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 333/651] 10.222.0.89 () {56 vars in 1108 bytes} [Thu Jul 28 17:31:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:31:42.179987 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22064cd4-79b6-4404-8763-4e1719164104 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:42.233241 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 319/652] 10.222.0.89 () {58 vars in 1282 bytes} [Thu Jul 28 17:31:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 17:31:42.233241 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba6c13a-6507-4a78-8bf9-1dbcdc68a0c4 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:42.273278 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:42.299708 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22064cd4-79b6-4404-8763-4e1719164104 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:42.340629 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e9ec93-d479-4f58-9d23-5a7f690103a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:42.341323 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e9ec93-d479-4f58-9d23-5a7f690103a7 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:42.341632 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e9ec93-d479-4f58-9d23-5a7f690103a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:42.342055 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba6c13a-6507-4a78-8bf9-1dbcdc68a0c4 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:42.342163 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e9ec93-d479-4f58-9d23-5a7f690103a7 None admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:42.342851 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e9ec93-d479-4f58-9d23-5a7f690103a7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:42.343179 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e9ec93-d479-4f58-9d23-5a7f690103a7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:42.344657 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e9ec93-d479-4f58-9d23-5a7f690103a7 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:42.359069 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 334/653] 10.222.0.89 () {56 vars in 1148 bytes} [Thu Jul 28 17:31:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:42.359422 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e9ec93-d479-4f58-9d23-5a7f690103a7 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:42.382574 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1155ec85-d415-46e4-8c4c-118438fbe901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:42.383333 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1155ec85-d415-46e4-8c4c-118438fbe901 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:42.383681 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1155ec85-d415-46e4-8c4c-118438fbe901 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:42.384022 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1155ec85-d415-46e4-8c4c-118438fbe901 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:42.391947 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1155ec85-d415-46e4-8c4c-118438fbe901 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:42.393409 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1155ec85-d415-46e4-8c4c-118438fbe901 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:42.405695 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-1155ec85-d415-46e4-8c4c-118438fbe901 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 28 17:31:42.405695 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:42.405695 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:42.405695 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:42.405695 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:42.405695 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:42.405695 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:42.405695 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:42.405695 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:42.405695 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:42.405695 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:42.405695 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:42.405695 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 17:31:42.405695 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 17:31:42.405695 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 17:31:42.405695 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 17:31:42.406887 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:42.406887 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:42.406887 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 17:31:42.406887 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:31:42.406887 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 17:31:42.406887 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:31:42.406887 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:42.406887 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:42.406887 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:42.406887 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:42.406887 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:42.406887 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:42.406887 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:42.406887 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:42.406887 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:42.406887 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:42.406887 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:42.406887 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:42.407934 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:42.407934 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:42.407934 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 17:31:42.407934 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 17:31:42.407934 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 17:31:42.407934 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 17:31:42.407934 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 17:31:42.407934 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 17:31:42.407934 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 28 17:31:42.407934 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:31:42.407934 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 320/654] 10.222.0.89 () {56 vars in 1099 bytes} [Thu Jul 28 17:31:42 2022] GET /identity/v3/users/glance => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:31:42.407934 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1155ec85-d415-46e4-8c4c-118438fbe901 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:42.423628 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:42.476313 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e9ec93-d479-4f58-9d23-5a7f690103a7 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:42.511902 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30073f2b-02fb-4bc7-b424-94d448cf73ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:42.512522 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30073f2b-02fb-4bc7-b424-94d448cf73ff None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:42.512800 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30073f2b-02fb-4bc7-b424-94d448cf73ff None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:42.513073 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30073f2b-02fb-4bc7-b424-94d448cf73ff None admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:42.514315 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30073f2b-02fb-4bc7-b424-94d448cf73ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:42.514618 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30073f2b-02fb-4bc7-b424-94d448cf73ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:42.514972 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1155ec85-d415-46e4-8c4c-118438fbe901 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:42.516000 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30073f2b-02fb-4bc7-b424-94d448cf73ff None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:42.528199 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 335/655] 10.222.0.89 () {56 vars in 1109 bytes} [Thu Jul 28 17:31:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:42.528631 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30073f2b-02fb-4bc7-b424-94d448cf73ff None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:42.531992 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:42.556356 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c7e607-59a9-40e0-9ae2-ac757f5e0f0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:42.557019 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c7e607-59a9-40e0-9ae2-ac757f5e0f0b None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:42.557372 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c7e607-59a9-40e0-9ae2-ac757f5e0f0b None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:42.557648 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c7e607-59a9-40e0-9ae2-ac757f5e0f0b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:42.568412 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c7e607-59a9-40e0-9ae2-ac757f5e0f0b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:42.569878 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c7e607-59a9-40e0-9ae2-ac757f5e0f0b None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:42.584313 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-30c7e607-59a9-40e0-9ae2-ac757f5e0f0b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:31:42.584313 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:42.584313 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:42.584313 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:42.584313 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:42.584313 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:42.584313 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:42.584313 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:42.584313 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:42.584313 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:42.584313 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:42.584313 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:42.584313 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:31:42.584313 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:31:42.584313 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:31:42.585466 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:31:42.585466 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:42.585466 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:42.585466 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:42.585466 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:42.585466 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:42.585466 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:42.585466 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:42.585466 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:42.585466 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:42.585466 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:42.585466 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:42.585466 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:42.585466 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:42.585466 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:42.585466 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:42.585466 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:42.586768 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:31:42.586768 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:31:42.586768 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:31:42.586768 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:31:42.586768 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 321/656] 10.222.0.89 () {56 vars in 1108 bytes} [Thu Jul 28 17:31:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:31:42.586768 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c7e607-59a9-40e0-9ae2-ac757f5e0f0b None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:42.633760 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30073f2b-02fb-4bc7-b424-94d448cf73ff None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:42.645646 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:42.667821 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5b3bd6-1940-45b0-907a-469ee11beed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:42.668431 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5b3bd6-1940-45b0-907a-469ee11beed3 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:42.668713 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5b3bd6-1940-45b0-907a-469ee11beed3 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:42.668993 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5b3bd6-1940-45b0-907a-469ee11beed3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:42.669810 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5b3bd6-1940-45b0-907a-469ee11beed3 None admin] RBAC: Adding query filter params (role.id=0a9ea6e6d6f94955a5dc949b0746a3f1, scope.project.id=04434e8535a64044996316f1d17dc0bc, user.id=d10b49b29df94404a71f43273839c252) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:42.670171 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5b3bd6-1940-45b0-907a-469ee11beed3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:42.671607 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5b3bd6-1940-45b0-907a-469ee11beed3 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:42.677600 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 336/657] 10.222.0.89 () {56 vars in 1374 bytes} [Thu Jul 28 17:31:42 2022] GET /identity/v3/role_assignments?user.id=d10b49b29df94404a71f43273839c252&scope.project.id=04434e8535a64044996316f1d17dc0bc&role.id=0a9ea6e6d6f94955a5dc949b0746a3f1 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:42.678546 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73b6515-09a7-48db-a349-4bcc7f2550ba None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:42.678828 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73b6515-09a7-48db-a349-4bcc7f2550ba None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:42.679105 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73b6515-09a7-48db-a349-4bcc7f2550ba None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:42.681210 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 337/658] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:31:42 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:31:42.681210 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73b6515-09a7-48db-a349-4bcc7f2550ba None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:42.693694 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c7e607-59a9-40e0-9ae2-ac757f5e0f0b None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:42.741674 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d53bbac-c55b-498f-94a8-95670ddf453e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:42.742614 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d53bbac-c55b-498f-94a8-95670ddf453e None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:42.742752 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d53bbac-c55b-498f-94a8-95670ddf453e None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:42.742934 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d53bbac-c55b-498f-94a8-95670ddf453e None admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:42.743754 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d53bbac-c55b-498f-94a8-95670ddf453e None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:42.744060 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d53bbac-c55b-498f-94a8-95670ddf453e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:42.745809 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d53bbac-c55b-498f-94a8-95670ddf453e None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:42.763051 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:42.770619 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 322/659] 10.222.0.89 () {56 vars in 1137 bytes} [Thu Jul 28 17:31:42 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:42.770619 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d53bbac-c55b-498f-94a8-95670ddf453e None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:42.808877 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73b6515-09a7-48db-a349-4bcc7f2550ba None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:42.856085 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da16760f-4431-4a4e-aa85-cde69ba47b81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:42.856757 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da16760f-4431-4a4e-aa85-cde69ba47b81 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:42.857041 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da16760f-4431-4a4e-aa85-cde69ba47b81 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:42.857454 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da16760f-4431-4a4e-aa85-cde69ba47b81 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:42.865385 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da16760f-4431-4a4e-aa85-cde69ba47b81 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:42.866838 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da16760f-4431-4a4e-aa85-cde69ba47b81 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:42.880000 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-da16760f-4431-4a4e-aa85-cde69ba47b81 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 28 17:31:42.880000 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:42.880000 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:42.880000 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:42.880000 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:42.880000 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:42.880000 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:42.880000 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:42.880000 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:42.880000 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:42.880000 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:42.880000 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:42.880000 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 17:31:42.880000 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 17:31:42.880000 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 17:31:42.880000 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 17:31:42.881236 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:42.881236 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:42.881236 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 17:31:42.881236 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:31:42.881236 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 17:31:42.881236 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:31:42.881236 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:42.881236 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:42.881236 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:42.881236 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:42.881236 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:42.881236 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:42.881236 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:42.881236 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:42.881236 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:42.881236 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:42.881236 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:42.881236 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:42.882512 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:42.882512 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:42.882512 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 17:31:42.882512 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 17:31:42.882512 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 17:31:42.882512 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 17:31:42.882512 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 17:31:42.882512 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 17:31:42.882512 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 28 17:31:42.882512 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:31:42.882512 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 338/660] 10.222.0.89 () {56 vars in 1102 bytes} [Thu Jul 28 17:31:42 2022] GET /identity/v3/users/neutron => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:31:42.887042 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85c127a-33d0-4991-9429-ef325249e789 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:42.887414 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85c127a-33d0-4991-9429-ef325249e789 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:42.887835 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85c127a-33d0-4991-9429-ef325249e789 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:42.896725 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d53bbac-c55b-498f-94a8-95670ddf453e None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:42.907731 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-e85c127a-33d0-4991-9429-ef325249e789 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:31:42.908973 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-e85c127a-33d0-4991-9429-ef325249e789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:31:42.916721 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:42.936126 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57c4928-54c2-4f81-8820-7c2a682a92d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:42.936757 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57c4928-54c2-4f81-8820-7c2a682a92d3 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:42.937146 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57c4928-54c2-4f81-8820-7c2a682a92d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:42.937445 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57c4928-54c2-4f81-8820-7c2a682a92d3 None admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:42.938147 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57c4928-54c2-4f81-8820-7c2a682a92d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:42.938453 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57c4928-54c2-4f81-8820-7c2a682a92d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:42.939816 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57c4928-54c2-4f81-8820-7c2a682a92d3 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:42.952868 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 323/661] 10.222.0.89 () {56 vars in 1109 bytes} [Thu Jul 28 17:31:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:42.953712 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57c4928-54c2-4f81-8820-7c2a682a92d3 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:42.994760 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 339/662] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:31:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:31:42.995795 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85c127a-33d0-4991-9429-ef325249e789 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:43.057411 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:43.072434 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57c4928-54c2-4f81-8820-7c2a682a92d3 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:43.106277 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85c127a-33d0-4991-9429-ef325249e789 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:43.109279 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1284c5-5907-413d-85e9-6a2a03771684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:43.109902 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1284c5-5907-413d-85e9-6a2a03771684 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:43.110178 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1284c5-5907-413d-85e9-6a2a03771684 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:43.110450 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1284c5-5907-413d-85e9-6a2a03771684 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:43.142851 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac7d5f3-09cf-46da-b41a-ab04bc35f020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:43.143394 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac7d5f3-09cf-46da-b41a-ab04bc35f020 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:43.143677 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac7d5f3-09cf-46da-b41a-ab04bc35f020 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:43.143949 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac7d5f3-09cf-46da-b41a-ab04bc35f020 None admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:43.144744 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac7d5f3-09cf-46da-b41a-ab04bc35f020 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:43.145050 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac7d5f3-09cf-46da-b41a-ab04bc35f020 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:43.146527 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac7d5f3-09cf-46da-b41a-ab04bc35f020 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:43.161608 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:43.169794 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 340/663] 10.222.0.89 () {56 vars in 1139 bytes} [Thu Jul 28 17:31:42 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:43.169794 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac7d5f3-09cf-46da-b41a-ab04bc35f020 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:43.191396 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1284c5-5907-413d-85e9-6a2a03771684 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:43.193181 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1284c5-5907-413d-85e9-6a2a03771684 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:43.281939 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac7d5f3-09cf-46da-b41a-ab04bc35f020 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:43.298584 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:43.304801 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 324/664] 10.222.0.89 () {58 vars in 1282 bytes} [Thu Jul 28 17:31:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 17:31:43.304801 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1284c5-5907-413d-85e9-6a2a03771684 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:43.327268 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19163896-164a-4939-a9d5-2dc3139ac6b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:43.327911 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19163896-164a-4939-a9d5-2dc3139ac6b3 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:43.328189 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19163896-164a-4939-a9d5-2dc3139ac6b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:43.328468 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19163896-164a-4939-a9d5-2dc3139ac6b3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:43.335378 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19163896-164a-4939-a9d5-2dc3139ac6b3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:43.336814 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19163896-164a-4939-a9d5-2dc3139ac6b3 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:43.349296 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-19163896-164a-4939-a9d5-2dc3139ac6b3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:31:43.349296 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:43.349296 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:43.349296 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:43.349296 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:43.349296 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:43.349296 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:43.349296 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:43.349296 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:43.349296 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:43.349296 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:43.349296 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:43.349296 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 17:31:43.349296 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 17:31:43.349296 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 17:31:43.350405 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 17:31:43.350405 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:43.350405 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:43.350405 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:43.350405 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:43.350405 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:43.350405 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:43.350405 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:43.350405 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:43.350405 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:43.350405 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:43.350405 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:43.350405 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:43.350405 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:43.350405 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:43.350405 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:43.350405 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:43.351294 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 17:31:43.351294 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 17:31:43.351294 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 17:31:43.351294 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 17:31:43.351294 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 17:31:43.351294 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 17:31:43.351294 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:31:43.351294 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:31:43.351294 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 341/665] 10.222.0.89 () {56 vars in 1111 bytes} [Thu Jul 28 17:31:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:31:43.351294 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19163896-164a-4939-a9d5-2dc3139ac6b3 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:43.396068 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:43.420816 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1284c5-5907-413d-85e9-6a2a03771684 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:43.457333 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088cc532-a44b-457c-be36-ea886f11380e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:43.458240 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088cc532-a44b-457c-be36-ea886f11380e None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:43.458240 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088cc532-a44b-457c-be36-ea886f11380e None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:43.458508 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088cc532-a44b-457c-be36-ea886f11380e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:43.461788 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19163896-164a-4939-a9d5-2dc3139ac6b3 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:43.469467 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088cc532-a44b-457c-be36-ea886f11380e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:31:43.474400 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-088cc532-a44b-457c-be36-ea886f11380e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 17:31:43.474400 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:43.474400 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:43.474400 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:43.474400 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:43.474400 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:43.474400 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:43.474400 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:43.474400 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:43.474400 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:43.474400 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:43.474400 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:43.474400 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:31:43.474400 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:31:43.474400 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:31:43.474400 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:31:43.475356 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:31:43.475356 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:31:43.475356 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:31:43.475356 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:31:43.475356 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:43.475356 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:43.475356 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:43.475356 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:43.475356 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:43.475356 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:43.475356 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:43.475356 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:43.475356 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:43.475356 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:43.475356 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:43.475356 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:43.476228 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:43.476228 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:43.476228 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:43.476228 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:43.476228 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:31:43.476228 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:31:43.476228 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:31:43.476228 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:31:43.476228 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:31:43.476228 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:31:43.476228 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 17:31:43.476228 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:31:43.476228 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 325/666] 10.222.0.89 () {56 vars in 1102 bytes} [Thu Jul 28 17:31:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:31:43.476939 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088cc532-a44b-457c-be36-ea886f11380e None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:43.502584 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c22e56-9e4e-42a6-a78b-f1971f4c3d8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:43.503201 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c22e56-9e4e-42a6-a78b-f1971f4c3d8f None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:43.503483 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c22e56-9e4e-42a6-a78b-f1971f4c3d8f None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:43.503754 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c22e56-9e4e-42a6-a78b-f1971f4c3d8f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:43.549559 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:43.578462 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c22e56-9e4e-42a6-a78b-f1971f4c3d8f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:43.580310 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c22e56-9e4e-42a6-a78b-f1971f4c3d8f None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:43.588082 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088cc532-a44b-457c-be36-ea886f11380e None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:43.626982 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d177fe9c-aef2-403e-801a-fea02b8e8bd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:43.628611 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d177fe9c-aef2-403e-801a-fea02b8e8bd0 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:43.629099 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d177fe9c-aef2-403e-801a-fea02b8e8bd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:43.629635 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d177fe9c-aef2-403e-801a-fea02b8e8bd0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:43.637621 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:43.642463 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d177fe9c-aef2-403e-801a-fea02b8e8bd0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:43.644866 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d177fe9c-aef2-403e-801a-fea02b8e8bd0 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:43.664580 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-d177fe9c-aef2-403e-801a-fea02b8e8bd0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:31:43.664580 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:43.664580 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:43.664580 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:43.664580 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:43.664580 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:43.664580 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:43.664580 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:43.664580 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:43.664580 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:43.664580 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:43.664580 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:43.664580 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:31:43.664580 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:31:43.664580 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:31:43.665532 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:31:43.665532 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:43.665532 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:43.665532 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:43.665532 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:43.665532 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:43.665532 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:43.665532 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:43.665532 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:43.665532 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:43.665532 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:43.665532 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:43.665532 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:43.665532 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:43.665532 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:43.665532 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:43.665532 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:43.666447 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:31:43.666447 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:31:43.666447 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:31:43.666447 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:31:43.666447 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 326/667] 10.222.0.89 () {56 vars in 1108 bytes} [Thu Jul 28 17:31:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:31:43.667182 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d177fe9c-aef2-403e-801a-fea02b8e8bd0 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:43.693314 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 342/668] 10.222.0.89 () {58 vars in 1282 bytes} [Thu Jul 28 17:31:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 17:31:43.694283 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c22e56-9e4e-42a6-a78b-f1971f4c3d8f None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:43.775123 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:43.783328 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d177fe9c-aef2-403e-801a-fea02b8e8bd0 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:43.802768 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c22e56-9e4e-42a6-a78b-f1971f4c3d8f None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:43.821622 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f837bf-7acc-4f23-826e-f04bfc11c428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:43.822246 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f837bf-7acc-4f23-826e-f04bfc11c428 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:43.822518 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f837bf-7acc-4f23-826e-f04bfc11c428 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:43.822914 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f837bf-7acc-4f23-826e-f04bfc11c428 None admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:43.826532 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f837bf-7acc-4f23-826e-f04bfc11c428 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:43.826532 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f837bf-7acc-4f23-826e-f04bfc11c428 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:43.826532 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f837bf-7acc-4f23-826e-f04bfc11c428 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:43.843414 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9075c6c-cd96-4c43-89b6-9bc7208caa69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:43.843821 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 327/669] 10.222.0.89 () {56 vars in 1148 bytes} [Thu Jul 28 17:31:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:43.844663 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f837bf-7acc-4f23-826e-f04bfc11c428 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:43.845770 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9075c6c-cd96-4c43-89b6-9bc7208caa69 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:43.846272 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9075c6c-cd96-4c43-89b6-9bc7208caa69 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:43.846786 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9075c6c-cd96-4c43-89b6-9bc7208caa69 None admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:43.848982 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9075c6c-cd96-4c43-89b6-9bc7208caa69 None admin] RBAC: Adding query filter params (name=service) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:43.849544 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9075c6c-cd96-4c43-89b6-9bc7208caa69 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:43.851133 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9075c6c-cd96-4c43-89b6-9bc7208caa69 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:43.862003 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 343/670] 10.222.0.89 () {56 vars in 1103 bytes} [Thu Jul 28 17:31:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:43.862751 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9075c6c-cd96-4c43-89b6-9bc7208caa69 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:43.907994 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:43.967964 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f837bf-7acc-4f23-826e-f04bfc11c428 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:43.979325 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9075c6c-cd96-4c43-89b6-9bc7208caa69 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:44.016749 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d234b5a2-c0ee-4f5c-9152-259dd87ecf8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:44.017580 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d234b5a2-c0ee-4f5c-9152-259dd87ecf8b None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:44.017954 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d234b5a2-c0ee-4f5c-9152-259dd87ecf8b None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:44.018326 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d234b5a2-c0ee-4f5c-9152-259dd87ecf8b None admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:44.018794 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f48bdf-9c6c-41f5-9117-87021adcd681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:44.019228 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d234b5a2-c0ee-4f5c-9152-259dd87ecf8b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:44.019303 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d234b5a2-c0ee-4f5c-9152-259dd87ecf8b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:44.019486 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f48bdf-9c6c-41f5-9117-87021adcd681 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:44.019696 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f48bdf-9c6c-41f5-9117-87021adcd681 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:44.019965 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f48bdf-9c6c-41f5-9117-87021adcd681 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:44.020872 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d234b5a2-c0ee-4f5c-9152-259dd87ecf8b None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:44.023011 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:44.033660 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f48bdf-9c6c-41f5-9117-87021adcd681 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:44.039112 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 328/671] 10.222.0.89 () {56 vars in 1109 bytes} [Thu Jul 28 17:31:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:44.039253 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f48bdf-9c6c-41f5-9117-87021adcd681 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:44.039333 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d234b5a2-c0ee-4f5c-9152-259dd87ecf8b None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:44.055676 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-71f48bdf-9c6c-41f5-9117-87021adcd681 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:31:44.055676 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:44.055676 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:44.055676 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:44.055676 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:44.055676 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:44.055676 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:44.055676 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:44.055676 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:44.055676 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:44.055676 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:44.055676 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:44.055676 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:31:44.055676 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:31:44.055676 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:31:44.056611 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:31:44.056611 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:44.056611 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:44.056611 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:44.056611 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:44.056611 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:44.056611 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:44.056611 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:44.056611 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:44.056611 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:44.056611 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:44.056611 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:44.056611 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:44.056611 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:44.056611 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:44.056611 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:44.056611 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:44.058328 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:31:44.058328 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:31:44.058328 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:31:44.058328 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:31:44.058328 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 344/672] 10.222.0.89 () {56 vars in 1108 bytes} [Thu Jul 28 17:31:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:31:44.058328 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f48bdf-9c6c-41f5-9117-87021adcd681 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:44.149498 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d234b5a2-c0ee-4f5c-9152-259dd87ecf8b None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:44.161225 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:44.167031 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f48bdf-9c6c-41f5-9117-87021adcd681 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:44.187671 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a4f632-e15c-4edc-8027-693ca6f13005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:44.188290 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a4f632-e15c-4edc-8027-693ca6f13005 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:44.188563 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a4f632-e15c-4edc-8027-693ca6f13005 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:44.188833 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a4f632-e15c-4edc-8027-693ca6f13005 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:44.189664 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a4f632-e15c-4edc-8027-693ca6f13005 None admin] RBAC: Adding query filter params (role.id=0a9ea6e6d6f94955a5dc949b0746a3f1, scope.project.id=04434e8535a64044996316f1d17dc0bc, user.id=96d512d0c84240918dae6c4a3e8b7b79) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:44.189967 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a4f632-e15c-4edc-8027-693ca6f13005 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:44.191394 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a4f632-e15c-4edc-8027-693ca6f13005 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:44.197196 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 329/673] 10.222.0.89 () {56 vars in 1374 bytes} [Thu Jul 28 17:31:44 2022] GET /identity/v3/role_assignments?user.id=96d512d0c84240918dae6c4a3e8b7b79&scope.project.id=04434e8535a64044996316f1d17dc0bc&role.id=0a9ea6e6d6f94955a5dc949b0746a3f1 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:44.197987 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a4f632-e15c-4edc-8027-693ca6f13005 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:44.222370 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62ecf45-3813-4905-9aba-12f0e7e22502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:44.223046 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62ecf45-3813-4905-9aba-12f0e7e22502 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:44.223430 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62ecf45-3813-4905-9aba-12f0e7e22502 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:44.223820 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62ecf45-3813-4905-9aba-12f0e7e22502 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:44.239422 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62ecf45-3813-4905-9aba-12f0e7e22502 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:44.241857 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62ecf45-3813-4905-9aba-12f0e7e22502 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:44.264703 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-a62ecf45-3813-4905-9aba-12f0e7e22502 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:31:44.264703 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:44.264703 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:44.264703 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:44.264703 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:44.264703 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:44.264703 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:44.264703 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:44.264703 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:44.264703 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:44.264703 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:44.264703 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:44.264703 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:31:44.264703 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:31:44.264703 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:31:44.265875 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:31:44.265875 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:44.265875 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:44.265875 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:44.265875 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:44.265875 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:44.265875 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:44.265875 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:44.265875 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:44.265875 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:44.265875 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:44.265875 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:44.265875 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:44.265875 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:44.265875 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:44.265875 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:44.265875 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:44.267008 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:31:44.267008 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:31:44.267008 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:31:44.267008 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:31:44.272379 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 345/674] 10.222.0.89 () {56 vars in 1108 bytes} [Thu Jul 28 17:31:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:31:44.272379 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62ecf45-3813-4905-9aba-12f0e7e22502 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:44.331398 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:44.341314 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a4f632-e15c-4edc-8027-693ca6f13005 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:44.394206 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1afd12-8842-48f5-b81c-30fe7178f150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:44.395695 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1afd12-8842-48f5-b81c-30fe7178f150 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:44.396422 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1afd12-8842-48f5-b81c-30fe7178f150 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:44.396916 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1afd12-8842-48f5-b81c-30fe7178f150 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:44.407505 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62ecf45-3813-4905-9aba-12f0e7e22502 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:44.409238 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1afd12-8842-48f5-b81c-30fe7178f150 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:44.412016 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1afd12-8842-48f5-b81c-30fe7178f150 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:44.426587 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-ca1afd12-8842-48f5-b81c-30fe7178f150 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:31:44.426587 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:44.426587 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:44.426587 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:44.426587 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:44.426587 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:44.426587 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:44.426587 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:44.426587 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:44.426587 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:44.426587 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:44.426587 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:44.426587 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 17:31:44.426587 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 17:31:44.426587 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 17:31:44.427545 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 17:31:44.427545 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:44.427545 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:44.427545 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:44.427545 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:44.427545 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:44.427545 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:44.427545 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:44.427545 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:44.427545 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:44.427545 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:44.427545 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:44.427545 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:44.427545 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:44.427545 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:44.427545 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:44.427545 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:44.428480 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 17:31:44.428480 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 17:31:44.428480 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 17:31:44.428480 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 17:31:44.428480 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 17:31:44.428480 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 17:31:44.428480 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:31:44.428480 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:31:44.430500 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 330/675] 10.222.0.89 () {56 vars in 1111 bytes} [Thu Jul 28 17:31:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:31:44.431443 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1afd12-8842-48f5-b81c-30fe7178f150 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:44.453063 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a12d2e-aaab-4a8b-b0f6-f0f9f908b99f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:44.453751 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a12d2e-aaab-4a8b-b0f6-f0f9f908b99f None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:44.454032 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a12d2e-aaab-4a8b-b0f6-f0f9f908b99f None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:44.454304 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a12d2e-aaab-4a8b-b0f6-f0f9f908b99f None admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:44.455012 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a12d2e-aaab-4a8b-b0f6-f0f9f908b99f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:44.455318 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a12d2e-aaab-4a8b-b0f6-f0f9f908b99f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:44.456696 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a12d2e-aaab-4a8b-b0f6-f0f9f908b99f None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:44.470847 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:44.471192 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 346/676] 10.222.0.89 () {56 vars in 1109 bytes} [Thu Jul 28 17:31:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:44.473561 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a12d2e-aaab-4a8b-b0f6-f0f9f908b99f None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:44.547625 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1afd12-8842-48f5-b81c-30fe7178f150 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:44.589913 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd27744a-f5ff-40b6-85de-bb9130daca96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:44.590950 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd27744a-f5ff-40b6-85de-bb9130daca96 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:44.591337 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd27744a-f5ff-40b6-85de-bb9130daca96 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:44.591620 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd27744a-f5ff-40b6-85de-bb9130daca96 None admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:44.592669 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd27744a-f5ff-40b6-85de-bb9130daca96 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:44.592980 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd27744a-f5ff-40b6-85de-bb9130daca96 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:44.594495 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd27744a-f5ff-40b6-85de-bb9130daca96 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:44.598177 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:44.601343 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a12d2e-aaab-4a8b-b0f6-f0f9f908b99f None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:44.610638 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 331/677] 10.222.0.89 () {56 vars in 1109 bytes} [Thu Jul 28 17:31:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:44.611576 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd27744a-f5ff-40b6-85de-bb9130daca96 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:44.639567 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7664d5e-3250-413d-829c-df14aaddf5b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:44.640669 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7664d5e-3250-413d-829c-df14aaddf5b2 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:44.641065 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7664d5e-3250-413d-829c-df14aaddf5b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:44.641685 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7664d5e-3250-413d-829c-df14aaddf5b2 None admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:44.642517 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7664d5e-3250-413d-829c-df14aaddf5b2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:44.642897 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7664d5e-3250-413d-829c-df14aaddf5b2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:44.644415 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7664d5e-3250-413d-829c-df14aaddf5b2 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:44.657525 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 347/678] 10.222.0.89 () {56 vars in 1148 bytes} [Thu Jul 28 17:31:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:44.658470 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7664d5e-3250-413d-829c-df14aaddf5b2 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:44.729247 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:44.739387 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd27744a-f5ff-40b6-85de-bb9130daca96 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:44.788051 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7664d5e-3250-413d-829c-df14aaddf5b2 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:44.788386 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db6f5df-1ef5-4086-9e70-8b8c4436192c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:44.789048 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db6f5df-1ef5-4086-9e70-8b8c4436192c None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:44.789357 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db6f5df-1ef5-4086-9e70-8b8c4436192c None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:44.789633 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db6f5df-1ef5-4086-9e70-8b8c4436192c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:44.799440 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db6f5df-1ef5-4086-9e70-8b8c4436192c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:44.800945 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db6f5df-1ef5-4086-9e70-8b8c4436192c None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:44.816140 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-5db6f5df-1ef5-4086-9e70-8b8c4436192c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:31:44.816140 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:44.816140 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:44.816140 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:44.816140 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:44.816140 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:44.816140 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:44.816140 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:44.816140 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:44.816140 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:44.816140 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:44.816140 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:44.816140 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 17:31:44.816140 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 17:31:44.816140 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 17:31:44.817586 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 17:31:44.817586 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:44.817586 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:44.817586 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:44.817586 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:44.817586 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:44.817586 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:44.817586 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:44.817586 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:44.817586 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:44.817586 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:44.817586 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:44.817586 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:44.817586 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:44.817586 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:44.817586 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:44.817586 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:44.818489 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 17:31:44.818489 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 17:31:44.818489 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 17:31:44.818489 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 17:31:44.818489 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 17:31:44.818489 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 17:31:44.818489 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:31:44.818489 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:31:44.818489 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 332/679] 10.222.0.89 () {56 vars in 1111 bytes} [Thu Jul 28 17:31:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:31:44.818489 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db6f5df-1ef5-4086-9e70-8b8c4436192c None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:44.839562 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501df62e-a75c-495c-a21d-2531795cafa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:44.840206 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501df62e-a75c-495c-a21d-2531795cafa5 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:44.840485 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501df62e-a75c-495c-a21d-2531795cafa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:44.840759 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501df62e-a75c-495c-a21d-2531795cafa5 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:44.850174 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501df62e-a75c-495c-a21d-2531795cafa5 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:44.851940 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501df62e-a75c-495c-a21d-2531795cafa5 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:44.867936 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-501df62e-a75c-495c-a21d-2531795cafa5 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 28 17:31:44.867936 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:44.867936 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:44.867936 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:44.867936 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:44.867936 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:44.867936 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:44.867936 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:44.867936 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:44.867936 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:44.867936 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:44.867936 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:44.867936 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 17:31:44.867936 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 17:31:44.867936 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 17:31:44.867936 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 17:31:44.869131 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:44.869131 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:44.869131 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 17:31:44.869131 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:31:44.869131 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 17:31:44.869131 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:31:44.869131 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:44.869131 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:44.869131 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:44.869131 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:44.869131 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:44.869131 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:44.869131 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:44.869131 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:44.869131 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:44.869131 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:44.869131 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:44.869131 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:44.870479 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:44.870479 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:44.870479 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 17:31:44.870479 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 17:31:44.870479 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 17:31:44.870479 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 17:31:44.870479 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 17:31:44.870479 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 17:31:44.870479 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 28 17:31:44.870479 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:31:44.873322 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 348/680] 10.222.0.89 () {56 vars in 1096 bytes} [Thu Jul 28 17:31:44 2022] GET /identity/v3/users/swift => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:31:44.875479 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d714835-5ac5-451d-970f-fecc91d94ee1 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:44.875795 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d714835-5ac5-451d-970f-fecc91d94ee1 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:44.876038 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d714835-5ac5-451d-970f-fecc91d94ee1 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:44.877626 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 349/681] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:31:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:31:44.878188 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d714835-5ac5-451d-970f-fecc91d94ee1 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:44.902238 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:44.950981 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db6f5df-1ef5-4086-9e70-8b8c4436192c None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:44.991561 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1fa030-0938-49aa-b5b8-a474e7907e6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:44.992092 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1fa030-0938-49aa-b5b8-a474e7907e6c None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:44.992372 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1fa030-0938-49aa-b5b8-a474e7907e6c None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:44.992644 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1fa030-0938-49aa-b5b8-a474e7907e6c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:44.993190 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d714835-5ac5-451d-970f-fecc91d94ee1 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:44.993483 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1fa030-0938-49aa-b5b8-a474e7907e6c None admin] RBAC: Adding query filter params (role.id=0a9ea6e6d6f94955a5dc949b0746a3f1, scope.project.id=04434e8535a64044996316f1d17dc0bc, user.id=1564db0771224a05b47d8e3cf778c147) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:44.993795 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1fa030-0938-49aa-b5b8-a474e7907e6c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:44.995514 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1fa030-0938-49aa-b5b8-a474e7907e6c None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:45.013198 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 333/682] 10.222.0.89 () {56 vars in 1374 bytes} [Thu Jul 28 17:31:44 2022] GET /identity/v3/role_assignments?user.id=1564db0771224a05b47d8e3cf778c147&scope.project.id=04434e8535a64044996316f1d17dc0bc&role.id=0a9ea6e6d6f94955a5dc949b0746a3f1 => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:45.013198 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1fa030-0938-49aa-b5b8-a474e7907e6c None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:45.055649 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b0d14c-61c2-46d2-981c-b57c91960579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:45.057909 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b0d14c-61c2-46d2-981c-b57c91960579 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:45.058286 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b0d14c-61c2-46d2-981c-b57c91960579 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:45.058656 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b0d14c-61c2-46d2-981c-b57c91960579 None admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:45.059543 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b0d14c-61c2-46d2-981c-b57c91960579 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:45.059970 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b0d14c-61c2-46d2-981c-b57c91960579 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:45.061870 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b0d14c-61c2-46d2-981c-b57c91960579 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:45.097328 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 350/683] 10.222.0.89 () {56 vars in 1148 bytes} [Thu Jul 28 17:31:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:45.098802 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376bb244-cfd2-4dd3-a02d-f0d3efb7e8e3 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:45.099182 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376bb244-cfd2-4dd3-a02d-f0d3efb7e8e3 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:45.099551 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376bb244-cfd2-4dd3-a02d-f0d3efb7e8e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:45.121822 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-376bb244-cfd2-4dd3-a02d-f0d3efb7e8e3 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:31:45.124397 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-376bb244-cfd2-4dd3-a02d-f0d3efb7e8e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:31:45.148629 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:45.237133 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 351/684] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:31:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:31:45.239625 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376bb244-cfd2-4dd3-a02d-f0d3efb7e8e3 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:45.252885 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1fa030-0938-49aa-b5b8-a474e7907e6c None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:45.260281 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:45.301247 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc91ad36-aebd-46e2-abe8-54863a02a6be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:45.301832 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc91ad36-aebd-46e2-abe8-54863a02a6be None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:45.302112 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc91ad36-aebd-46e2-abe8-54863a02a6be None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:45.302387 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc91ad36-aebd-46e2-abe8-54863a02a6be None admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:45.303186 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc91ad36-aebd-46e2-abe8-54863a02a6be None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:45.303546 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc91ad36-aebd-46e2-abe8-54863a02a6be None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:45.305195 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc91ad36-aebd-46e2-abe8-54863a02a6be None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:45.329570 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 334/685] 10.222.0.89 () {56 vars in 1135 bytes} [Thu Jul 28 17:31:45 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:45.330655 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc91ad36-aebd-46e2-abe8-54863a02a6be None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:45.387558 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376bb244-cfd2-4dd3-a02d-f0d3efb7e8e3 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:45.428015 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:45.438392 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d87220-1ea7-492c-87a4-0f02c6b04c13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:45.440109 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d87220-1ea7-492c-87a4-0f02c6b04c13 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:45.440388 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d87220-1ea7-492c-87a4-0f02c6b04c13 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:45.441691 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d87220-1ea7-492c-87a4-0f02c6b04c13 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:45.447350 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d87220-1ea7-492c-87a4-0f02c6b04c13 None admin] RBAC: Adding query filter params (role.id=0a9ea6e6d6f94955a5dc949b0746a3f1, scope.project.id=04434e8535a64044996316f1d17dc0bc, user.id=76817cd541294331b0023069092e8e89) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:45.447673 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d87220-1ea7-492c-87a4-0f02c6b04c13 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:45.449235 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d87220-1ea7-492c-87a4-0f02c6b04c13 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:45.456229 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc91ad36-aebd-46e2-abe8-54863a02a6be None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:45.456581 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 352/686] 10.222.0.89 () {56 vars in 1374 bytes} [Thu Jul 28 17:31:45 2022] GET /identity/v3/role_assignments?user.id=76817cd541294331b0023069092e8e89&scope.project.id=04434e8535a64044996316f1d17dc0bc&role.id=0a9ea6e6d6f94955a5dc949b0746a3f1 => generated 608 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:45.461620 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d87220-1ea7-492c-87a4-0f02c6b04c13 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:45.503597 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f94c59b-462a-496a-ae97-ba7c50f49404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:45.504720 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f94c59b-462a-496a-ae97-ba7c50f49404 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:45.505257 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f94c59b-462a-496a-ae97-ba7c50f49404 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:45.505805 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f94c59b-462a-496a-ae97-ba7c50f49404 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:45.527720 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f94c59b-462a-496a-ae97-ba7c50f49404 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:31:45.533539 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-4f94c59b-462a-496a-ae97-ba7c50f49404 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:31:45.533539 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:45.533539 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:45.533539 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:45.533539 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:45.533539 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:45.533539 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:45.533539 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:45.533539 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:45.533539 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:45.533539 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:45.533539 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:45.533539 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:31:45.533539 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:31:45.533539 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:31:45.533539 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:31:45.534539 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:31:45.534539 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:31:45.534539 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:31:45.534539 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:31:45.534539 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:45.534539 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:45.534539 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:45.534539 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:45.534539 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:45.534539 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:45.534539 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:45.534539 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:45.534539 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:45.534539 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:45.534539 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:45.534539 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:45.535416 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:45.535416 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:45.535416 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:45.535416 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:45.535416 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:31:45.535416 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:31:45.535416 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:31:45.535416 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:31:45.535416 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:31:45.535416 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:31:45.535416 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:31:45.535416 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:31:45.538432 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 335/687] 10.222.0.89 () {56 vars in 1096 bytes} [Thu Jul 28 17:31:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:31:45.546606 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f94c59b-462a-496a-ae97-ba7c50f49404 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:45.671209 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:45.676407 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f94c59b-462a-496a-ae97-ba7c50f49404 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:45.723334 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c701f65c-e8b6-4981-a997-e847cd544c09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:45.723963 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c701f65c-e8b6-4981-a997-e847cd544c09 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:45.724241 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c701f65c-e8b6-4981-a997-e847cd544c09 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:45.724511 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c701f65c-e8b6-4981-a997-e847cd544c09 None admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:45.725268 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c701f65c-e8b6-4981-a997-e847cd544c09 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:45.725561 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c701f65c-e8b6-4981-a997-e847cd544c09 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:45.726944 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c701f65c-e8b6-4981-a997-e847cd544c09 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:45.729472 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:45.730464 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d87220-1ea7-492c-87a4-0f02c6b04c13 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:45.737384 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 336/688] 10.222.0.89 () {56 vars in 1099 bytes} [Thu Jul 28 17:31:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:45.742807 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c701f65c-e8b6-4981-a997-e847cd544c09 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:45.785008 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e50a7c-55ee-4269-92d9-7a71626592c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:45.785597 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e50a7c-55ee-4269-92d9-7a71626592c4 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:45.785880 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e50a7c-55ee-4269-92d9-7a71626592c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:45.786149 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e50a7c-55ee-4269-92d9-7a71626592c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:45.886121 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c701f65c-e8b6-4981-a997-e847cd544c09 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:45.887982 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e50a7c-55ee-4269-92d9-7a71626592c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:45.891592 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e50a7c-55ee-4269-92d9-7a71626592c4 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:45.939772 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8813aa1-a4ac-4228-8d9c-792744de50ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:45.940481 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8813aa1-a4ac-4228-8d9c-792744de50ae None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:45.940769 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8813aa1-a4ac-4228-8d9c-792744de50ae None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:45.941039 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8813aa1-a4ac-4228-8d9c-792744de50ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:45.955885 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8813aa1-a4ac-4228-8d9c-792744de50ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:45.957423 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8813aa1-a4ac-4228-8d9c-792744de50ae None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:45.976789 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-b8813aa1-a4ac-4228-8d9c-792744de50ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:31:45.976789 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:45.976789 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:45.976789 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:45.976789 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:45.976789 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:45.976789 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:45.976789 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:45.976789 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:45.976789 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:45.976789 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:45.976789 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:45.976789 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:31:45.976789 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:31:45.976789 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:31:45.977800 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:31:45.977800 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:45.977800 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:45.977800 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:45.977800 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:45.977800 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:45.977800 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:45.977800 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:45.977800 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:45.977800 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:45.977800 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:45.977800 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:45.977800 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:45.977800 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:45.977800 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:45.977800 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:45.977800 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:45.978702 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:31:45.978702 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:31:45.978702 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:31:45.978702 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:31:45.978702 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 337/689] 10.222.0.89 () {56 vars in 1108 bytes} [Thu Jul 28 17:31:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:31:45.984055 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8813aa1-a4ac-4228-8d9c-792744de50ae None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:45.989628 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:46.026030 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 353/690] 10.222.0.89 () {58 vars in 1282 bytes} [Thu Jul 28 17:31:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 569 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 17:31:46.033335 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e50a7c-55ee-4269-92d9-7a71626592c4 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:46.056832 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:46.120357 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8813aa1-a4ac-4228-8d9c-792744de50ae None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:46.156547 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ad5ab0-3f27-48c1-aa8d-26857f1540f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:46.157571 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ad5ab0-3f27-48c1-aa8d-26857f1540f8 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:46.158079 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ad5ab0-3f27-48c1-aa8d-26857f1540f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:46.158551 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ad5ab0-3f27-48c1-aa8d-26857f1540f8 None admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:46.159449 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ad5ab0-3f27-48c1-aa8d-26857f1540f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:46.159952 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ad5ab0-3f27-48c1-aa8d-26857f1540f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:46.161625 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ad5ab0-3f27-48c1-aa8d-26857f1540f8 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:46.175730 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 338/691] 10.222.0.89 () {56 vars in 1109 bytes} [Thu Jul 28 17:31:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:31:46.181737 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ad5ab0-3f27-48c1-aa8d-26857f1540f8 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:46.232126 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e50a7c-55ee-4269-92d9-7a71626592c4 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:46.269676 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03e50a9-bf60-49c4-bb9b-8a1285db596c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:46.270319 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03e50a9-bf60-49c4-bb9b-8a1285db596c None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:46.270599 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03e50a9-bf60-49c4-bb9b-8a1285db596c None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:46.270874 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03e50a9-bf60-49c4-bb9b-8a1285db596c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:46.282808 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03e50a9-bf60-49c4-bb9b-8a1285db596c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:46.284453 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03e50a9-bf60-49c4-bb9b-8a1285db596c None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:46.296124 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ad5ab0-3f27-48c1-aa8d-26857f1540f8 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:46.300654 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-a03e50a9-bf60-49c4-bb9b-8a1285db596c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:31:46.300654 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:46.300654 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:46.300654 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:46.300654 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:46.300654 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:46.300654 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:46.300654 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:46.300654 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:46.300654 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:46.300654 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:46.300654 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:46.300654 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:31:46.300654 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:31:46.300654 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:31:46.301724 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:31:46.301724 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:46.301724 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:46.301724 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:46.301724 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:46.301724 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:46.301724 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:46.301724 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:46.301724 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:46.301724 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:46.301724 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:46.301724 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:46.301724 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:46.301724 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:46.301724 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:46.301724 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:46.301724 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:46.302705 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:31:46.302705 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:31:46.302705 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:31:46.302705 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:31:46.302705 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 354/692] 10.222.0.89 () {56 vars in 1108 bytes} [Thu Jul 28 17:31:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:31:46.307673 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03e50a9-bf60-49c4-bb9b-8a1285db596c None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:46.311450 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:46.337668 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:46.338771 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e419bde-51cb-4a09-97ca-faae7828c606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:46.339816 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e419bde-51cb-4a09-97ca-faae7828c606 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:46.340302 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e419bde-51cb-4a09-97ca-faae7828c606 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:46.340771 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e419bde-51cb-4a09-97ca-faae7828c606 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:46.349437 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e419bde-51cb-4a09-97ca-faae7828c606 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:46.351171 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e419bde-51cb-4a09-97ca-faae7828c606 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:46.363708 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-0e419bde-51cb-4a09-97ca-faae7828c606 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 17:31:46.363708 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:46.363708 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:46.363708 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:46.363708 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:46.363708 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:46.363708 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:46.363708 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:46.363708 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:46.363708 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:46.363708 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:46.363708 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:46.363708 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 17:31:46.363708 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 17:31:46.363708 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 17:31:46.363708 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 17:31:46.364702 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:46.364702 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:46.364702 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 17:31:46.364702 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:31:46.364702 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 17:31:46.364702 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:31:46.364702 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:46.364702 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:46.364702 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:46.364702 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:46.364702 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:46.364702 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:46.364702 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:46.364702 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:46.364702 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:46.364702 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:46.364702 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:46.364702 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:46.365693 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:46.365693 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:46.365693 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 17:31:46.365693 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 17:31:46.365693 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 17:31:46.365693 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 17:31:46.365693 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 17:31:46.365693 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 17:31:46.365693 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 17:31:46.365693 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:31:46.368230 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 339/693] 10.222.0.89 () {56 vars in 1093 bytes} [Thu Jul 28 17:31:46 2022] GET /identity/v3/users/nova => generated 82 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:31:46.377772 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e419bde-51cb-4a09-97ca-faae7828c606 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:46.417484 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03e50a9-bf60-49c4-bb9b-8a1285db596c None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:46.458249 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dea42b8-3fd3-46b5-9de3-547ae99893df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:46.458871 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dea42b8-3fd3-46b5-9de3-547ae99893df None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:46.459157 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dea42b8-3fd3-46b5-9de3-547ae99893df None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:46.459420 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dea42b8-3fd3-46b5-9de3-547ae99893df None admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:46.460113 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dea42b8-3fd3-46b5-9de3-547ae99893df None admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:46.460417 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dea42b8-3fd3-46b5-9de3-547ae99893df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:46.462430 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dea42b8-3fd3-46b5-9de3-547ae99893df None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:46.476166 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 355/694] 10.222.0.89 () {56 vars in 1109 bytes} [Thu Jul 28 17:31:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:46.481544 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dea42b8-3fd3-46b5-9de3-547ae99893df None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:46.577984 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:46.596013 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dea42b8-3fd3-46b5-9de3-547ae99893df None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:46.599220 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e419bde-51cb-4a09-97ca-faae7828c606 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:46.633415 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e36c3b-f853-4bb1-a3ae-6d2d0a611604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:46.634053 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e36c3b-f853-4bb1-a3ae-6d2d0a611604 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:46.634331 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e36c3b-f853-4bb1-a3ae-6d2d0a611604 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:46.634601 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e36c3b-f853-4bb1-a3ae-6d2d0a611604 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:46.642261 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e36c3b-f853-4bb1-a3ae-6d2d0a611604 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:46.643773 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e36c3b-f853-4bb1-a3ae-6d2d0a611604 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:46.644602 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35c4cf3-3811-4004-8895-9ca382d945a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:46.645686 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35c4cf3-3811-4004-8895-9ca382d945a2 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:46.646108 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35c4cf3-3811-4004-8895-9ca382d945a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:46.646954 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35c4cf3-3811-4004-8895-9ca382d945a2 None admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:46.647889 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35c4cf3-3811-4004-8895-9ca382d945a2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:46.648353 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35c4cf3-3811-4004-8895-9ca382d945a2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:46.650208 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35c4cf3-3811-4004-8895-9ca382d945a2 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:46.656311 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-b9e36c3b-f853-4bb1-a3ae-6d2d0a611604 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:31:46.656311 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:46.656311 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:46.656311 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:46.656311 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:46.656311 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:46.656311 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:46.656311 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:46.656311 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:46.656311 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:46.656311 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:46.656311 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:46.656311 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 17:31:46.656311 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 17:31:46.656311 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 17:31:46.657405 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 17:31:46.657405 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:46.657405 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:46.657405 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:46.657405 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:46.657405 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:46.657405 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:46.657405 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:46.657405 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:46.657405 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:46.657405 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:46.657405 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:46.657405 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:46.657405 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:46.657405 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:46.657405 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:46.657405 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:46.658596 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 17:31:46.658596 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 17:31:46.658596 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 17:31:46.658596 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 17:31:46.658596 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 17:31:46.658596 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 17:31:46.658596 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:31:46.658596 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:31:46.658596 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 356/695] 10.222.0.89 () {56 vars in 1111 bytes} [Thu Jul 28 17:31:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:31:46.663898 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796feb06-d509-4c89-91c0-b31024248d3d None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:46.664182 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796feb06-d509-4c89-91c0-b31024248d3d None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:46.664563 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796feb06-d509-4c89-91c0-b31024248d3d None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:46.665885 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 357/696] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:31:46 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:31:46.666584 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796feb06-d509-4c89-91c0-b31024248d3d None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:46.677394 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 340/697] 10.222.0.89 () {56 vars in 1133 bytes} [Thu Jul 28 17:31:46 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:46.678989 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b46b8e-6369-4c17-820c-fb680685bf4d None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:46.679430 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b46b8e-6369-4c17-820c-fb680685bf4d None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:46.679847 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b46b8e-6369-4c17-820c-fb680685bf4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:46.704228 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-17b46b8e-6369-4c17-820c-fb680685bf4d None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:31:46.706515 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-17b46b8e-6369-4c17-820c-fb680685bf4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:31:46.743979 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:46.785370 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796feb06-d509-4c89-91c0-b31024248d3d None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:46.796602 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 341/698] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:31:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:31:46.797770 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b46b8e-6369-4c17-820c-fb680685bf4d None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:46.824049 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2b9a14-9760-4dd9-b44f-379cbd1cfb98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:46.824902 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2b9a14-9760-4dd9-b44f-379cbd1cfb98 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:46.825477 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2b9a14-9760-4dd9-b44f-379cbd1cfb98 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:46.825914 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2b9a14-9760-4dd9-b44f-379cbd1cfb98 None admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:46.830012 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2b9a14-9760-4dd9-b44f-379cbd1cfb98 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:46.830483 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2b9a14-9760-4dd9-b44f-379cbd1cfb98 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:46.832196 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2b9a14-9760-4dd9-b44f-379cbd1cfb98 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:46.848840 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 358/699] 10.222.0.89 () {56 vars in 1148 bytes} [Thu Jul 28 17:31:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:46.849874 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2b9a14-9760-4dd9-b44f-379cbd1cfb98 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:46.870400 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:46.926083 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b46b8e-6369-4c17-820c-fb680685bf4d None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:46.965401 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9887118d-11e5-46b7-bfc3-556c97986a3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:46.966318 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9887118d-11e5-46b7-bfc3-556c97986a3f None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:46.966779 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9887118d-11e5-46b7-bfc3-556c97986a3f None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:46.967935 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9887118d-11e5-46b7-bfc3-556c97986a3f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:46.990024 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2b9a14-9760-4dd9-b44f-379cbd1cfb98 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:46.997440 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:47.046401 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734ca9cf-75fe-4eee-8ce3-89c3586a8088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:47.047333 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734ca9cf-75fe-4eee-8ce3-89c3586a8088 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:47.047821 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734ca9cf-75fe-4eee-8ce3-89c3586a8088 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:47.048286 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734ca9cf-75fe-4eee-8ce3-89c3586a8088 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:47.049108 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734ca9cf-75fe-4eee-8ce3-89c3586a8088 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:47.049303 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9887118d-11e5-46b7-bfc3-556c97986a3f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:47.050713 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9887118d-11e5-46b7-bfc3-556c97986a3f None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:47.051098 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734ca9cf-75fe-4eee-8ce3-89c3586a8088 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:47.063201 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-734ca9cf-75fe-4eee-8ce3-89c3586a8088 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 17:31:47.063201 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:47.063201 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 17:31:47.063201 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 17:31:47.063201 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 17:31:47.063201 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 17:31:47.063201 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 17:31:47.063201 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:31:47.063201 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 17:31:47.063201 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:31:47.063201 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 17:31:47.063201 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:31:47.063201 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:47.063201 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:47.063201 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:47.063201 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:47.064224 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:47.064224 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:47.064224 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:47.064224 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:47.064224 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:47.064224 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:47.064224 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:47.064224 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 17:31:47.064224 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 17:31:47.064224 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 17:31:47.064224 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 17:31:47.064224 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:47.064224 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:47.064224 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:47.064224 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:47.064224 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:47.065096 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:47.065096 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:47.065096 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:47.065096 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:47.065096 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:47.065096 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:47.065096 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:47.065096 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:47.065096 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:47.065096 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:47.065096 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:47.065096 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 17:31:47.065096 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:31:47.065096 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 17:31:47.065096 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:31:47.068071 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 359/700] 10.222.0.89 () {56 vars in 1129 bytes} [Thu Jul 28 17:31:46 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:31:47.069140 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734ca9cf-75fe-4eee-8ce3-89c3586a8088 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:47.147348 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:47.183162 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734ca9cf-75fe-4eee-8ce3-89c3586a8088 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:47.193253 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 342/701] 10.222.0.89 () {58 vars in 1282 bytes} [Thu Jul 28 17:31:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 17:31:47.194146 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9887118d-11e5-46b7-bfc3-556c97986a3f None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:47.222785 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681cbdee-ec88-4b43-bf8f-577e47a9ff15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:47.223397 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681cbdee-ec88-4b43-bf8f-577e47a9ff15 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:47.223673 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681cbdee-ec88-4b43-bf8f-577e47a9ff15 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:47.223942 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681cbdee-ec88-4b43-bf8f-577e47a9ff15 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:47.224725 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681cbdee-ec88-4b43-bf8f-577e47a9ff15 None admin] RBAC: Adding query filter params (role.id=0a9ea6e6d6f94955a5dc949b0746a3f1, scope.project.id=04434e8535a64044996316f1d17dc0bc, user.id=3c59473b01bf4433ba3b111114fc048b) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:47.225034 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681cbdee-ec88-4b43-bf8f-577e47a9ff15 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:47.226669 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681cbdee-ec88-4b43-bf8f-577e47a9ff15 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:47.234109 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 360/702] 10.222.0.89 () {56 vars in 1374 bytes} [Thu Jul 28 17:31:47 2022] GET /identity/v3/role_assignments?user.id=3c59473b01bf4433ba3b111114fc048b&scope.project.id=04434e8535a64044996316f1d17dc0bc&role.id=0a9ea6e6d6f94955a5dc949b0746a3f1 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:47.234915 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681cbdee-ec88-4b43-bf8f-577e47a9ff15 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:47.292738 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:47.375776 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9887118d-11e5-46b7-bfc3-556c97986a3f None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:47.439430 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b479fc-b5ac-4f4d-9c1f-b1b95422e583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:47.440068 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b479fc-b5ac-4f4d-9c1f-b1b95422e583 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:47.440346 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b479fc-b5ac-4f4d-9c1f-b1b95422e583 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:47.440619 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b479fc-b5ac-4f4d-9c1f-b1b95422e583 None admin] PATH_INFO: `/v3/services` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:47.441332 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b479fc-b5ac-4f4d-9c1f-b1b95422e583 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:47.441638 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b479fc-b5ac-4f4d-9c1f-b1b95422e583 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:47.443009 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b479fc-b5ac-4f4d-9c1f-b1b95422e583 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:47.452327 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 343/703] 10.222.0.89 () {56 vars in 1124 bytes} [Thu Jul 28 17:31:47 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:47.457365 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681cbdee-ec88-4b43-bf8f-577e47a9ff15 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:47.458970 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b479fc-b5ac-4f4d-9c1f-b1b95422e583 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:47.507528 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb8cdac-ed02-4663-8728-8ef7a6d4a6cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:47.508170 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb8cdac-ed02-4663-8728-8ef7a6d4a6cd None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:47.508449 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb8cdac-ed02-4663-8728-8ef7a6d4a6cd None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:47.508720 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb8cdac-ed02-4663-8728-8ef7a6d4a6cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:47.510756 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:47.523583 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb8cdac-ed02-4663-8728-8ef7a6d4a6cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:47.525043 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb8cdac-ed02-4663-8728-8ef7a6d4a6cd None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:47.545611 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-4fb8cdac-ed02-4663-8728-8ef7a6d4a6cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:31:47.545611 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:47.545611 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:47.545611 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:47.545611 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:47.545611 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:47.545611 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:47.545611 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:47.545611 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:47.545611 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:47.545611 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:47.545611 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:47.545611 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:31:47.545611 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:31:47.545611 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:31:47.546608 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:31:47.546608 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:47.546608 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:47.546608 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:47.546608 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:47.546608 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:47.546608 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:47.546608 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:47.546608 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:47.546608 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:47.546608 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:47.546608 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:47.546608 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:47.546608 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:47.546608 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:47.546608 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:47.546608 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:47.547637 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:31:47.547637 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:31:47.547637 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:31:47.547637 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:31:47.547637 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 361/704] 10.222.0.89 () {56 vars in 1108 bytes} [Thu Jul 28 17:31:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:31:47.547955 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cf27ae-8a37-49fd-9879-e7067cefde21 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:47.547955 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cf27ae-8a37-49fd-9879-e7067cefde21 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:47.548390 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cf27ae-8a37-49fd-9879-e7067cefde21 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:47.552645 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 362/705] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:31:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:31:47.553998 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cf27ae-8a37-49fd-9879-e7067cefde21 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:47.597931 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b479fc-b5ac-4f4d-9c1f-b1b95422e583 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:47.627875 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:47.644209 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3d91f0-d1e2-4836-92ce-eb593ad692d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:47.645208 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3d91f0-d1e2-4836-92ce-eb593ad692d7 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:47.645703 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3d91f0-d1e2-4836-92ce-eb593ad692d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:47.646205 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3d91f0-d1e2-4836-92ce-eb593ad692d7 None admin] PATH_INFO: `/v3/services` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:47.647096 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3d91f0-d1e2-4836-92ce-eb593ad692d7 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:47.647625 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3d91f0-d1e2-4836-92ce-eb593ad692d7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:47.649214 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3d91f0-d1e2-4836-92ce-eb593ad692d7 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:47.656203 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 344/706] 10.222.0.89 () {56 vars in 1124 bytes} [Thu Jul 28 17:31:47 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:47.657736 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908499af-40f5-418b-9127-8b581410d7af None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:47.658293 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908499af-40f5-418b-9127-8b581410d7af None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:47.658772 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908499af-40f5-418b-9127-8b581410d7af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:47.678345 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-908499af-40f5-418b-9127-8b581410d7af None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:31:47.679956 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-908499af-40f5-418b-9127-8b581410d7af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:31:47.738074 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cf27ae-8a37-49fd-9879-e7067cefde21 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:47.763363 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 345/707] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:31:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:31:47.796384 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cccc34-e807-4da0-b66d-f80875e29143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:47.797510 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cccc34-e807-4da0-b66d-f80875e29143 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:47.798025 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cccc34-e807-4da0-b66d-f80875e29143 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:47.798587 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cccc34-e807-4da0-b66d-f80875e29143 None admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:47.799532 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cccc34-e807-4da0-b66d-f80875e29143 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:47.800060 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cccc34-e807-4da0-b66d-f80875e29143 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:47.801763 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cccc34-e807-4da0-b66d-f80875e29143 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:47.808888 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908499af-40f5-418b-9127-8b581410d7af None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:47.838850 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 363/708] 10.222.0.89 () {56 vars in 1109 bytes} [Thu Jul 28 17:31:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:47.843166 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cccc34-e807-4da0-b66d-f80875e29143 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:47.937763 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:47.975292 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908499af-40f5-418b-9127-8b581410d7af None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:48.001861 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:48.018998 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cccc34-e807-4da0-b66d-f80875e29143 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:48.043482 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daf0878-e2d6-4733-8716-af022308ed8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:48.044440 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daf0878-e2d6-4733-8716-af022308ed8a None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:48.044737 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daf0878-e2d6-4733-8716-af022308ed8a None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:48.045075 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daf0878-e2d6-4733-8716-af022308ed8a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:48.069784 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8daf0878-e2d6-4733-8716-af022308ed8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:48.071296 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8daf0878-e2d6-4733-8716-af022308ed8a None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:48.095975 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4e8de4-f0ba-4313-aea4-bbf7b7661bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:48.096629 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4e8de4-f0ba-4313-aea4-bbf7b7661bc5 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:48.096935 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4e8de4-f0ba-4313-aea4-bbf7b7661bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:48.099275 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4e8de4-f0ba-4313-aea4-bbf7b7661bc5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:48.103503 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-8daf0878-e2d6-4733-8716-af022308ed8a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:31:48.103503 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:48.103503 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:48.103503 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:48.103503 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:48.103503 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:48.103503 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:48.103503 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:48.103503 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:48.103503 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:48.103503 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:48.103503 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:48.103503 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:31:48.103503 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:31:48.103503 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:31:48.104390 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:31:48.104390 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:48.104390 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:48.104390 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:48.104390 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:48.104390 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:48.104390 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:48.104390 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:48.104390 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:48.104390 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:48.104390 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:48.104390 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:48.104390 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:48.104390 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:48.104390 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:48.104390 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:48.104390 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:48.105699 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:31:48.105699 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:31:48.105699 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:31:48.105699 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:31:48.105699 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 346/709] 10.222.0.89 () {56 vars in 1108 bytes} [Thu Jul 28 17:31:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:31:48.112241 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daf0878-e2d6-4733-8716-af022308ed8a None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:48.116039 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4e8de4-f0ba-4313-aea4-bbf7b7661bc5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:48.117653 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4e8de4-f0ba-4313-aea4-bbf7b7661bc5 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:48.137152 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-2b4e8de4-f0ba-4313-aea4-bbf7b7661bc5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:31:48.137152 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:48.137152 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:48.137152 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:48.137152 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:48.137152 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:48.137152 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:48.137152 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:48.137152 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:48.137152 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:48.137152 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:48.137152 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:48.137152 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 17:31:48.137152 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 17:31:48.137152 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 17:31:48.138139 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 17:31:48.138139 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:48.138139 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:48.138139 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:48.138139 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:48.138139 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:48.138139 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:48.138139 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:48.138139 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:48.138139 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:48.138139 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:48.138139 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:48.138139 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:48.138139 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:48.138139 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:48.138139 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:48.138139 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:48.140002 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 17:31:48.140002 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 17:31:48.140002 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 17:31:48.140002 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 17:31:48.140002 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 17:31:48.140002 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 17:31:48.140002 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:31:48.140002 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:31:48.143719 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 364/710] 10.222.0.89 () {56 vars in 1111 bytes} [Thu Jul 28 17:31:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:31:48.149915 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4e8de4-f0ba-4313-aea4-bbf7b7661bc5 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:48.244291 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daf0878-e2d6-4733-8716-af022308ed8a None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:48.288718 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb9f525-1bea-4f93-a2cd-e2d232d39740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:48.289713 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb9f525-1bea-4f93-a2cd-e2d232d39740 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:48.290194 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb9f525-1bea-4f93-a2cd-e2d232d39740 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:48.290685 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb9f525-1bea-4f93-a2cd-e2d232d39740 None admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:48.291577 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb9f525-1bea-4f93-a2cd-e2d232d39740 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:48.292248 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb9f525-1bea-4f93-a2cd-e2d232d39740 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:48.293897 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb9f525-1bea-4f93-a2cd-e2d232d39740 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:48.307890 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4e8de4-f0ba-4313-aea4-bbf7b7661bc5 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:48.313336 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 347/711] 10.222.0.89 () {56 vars in 1109 bytes} [Thu Jul 28 17:31:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:48.314798 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c22a694-09b7-4c0f-92a4-694484711c37 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:48.315088 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c22a694-09b7-4c0f-92a4-694484711c37 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:48.315543 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:48.316096 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c22a694-09b7-4c0f-92a4-694484711c37 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:48.319057 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 348/712] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:31:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:31:48.319778 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c22a694-09b7-4c0f-92a4-694484711c37 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:48.351882 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:48.357253 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45438b05-92db-45db-885c-e5261d2e4755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:48.358654 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45438b05-92db-45db-885c-e5261d2e4755 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:48.358839 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45438b05-92db-45db-885c-e5261d2e4755 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:48.359103 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45438b05-92db-45db-885c-e5261d2e4755 None admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:48.360048 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45438b05-92db-45db-885c-e5261d2e4755 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:48.360355 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45438b05-92db-45db-885c-e5261d2e4755 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:48.362083 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45438b05-92db-45db-885c-e5261d2e4755 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:48.387889 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 365/713] 10.222.0.89 () {56 vars in 1148 bytes} [Thu Jul 28 17:31:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:31:48.387889 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74eb7d09-8f56-422e-97c8-4380b6df3cba None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:48.387889 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74eb7d09-8f56-422e-97c8-4380b6df3cba None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:48.387889 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74eb7d09-8f56-422e-97c8-4380b6df3cba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:48.406260 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-74eb7d09-8f56-422e-97c8-4380b6df3cba None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:31:48.407522 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-74eb7d09-8f56-422e-97c8-4380b6df3cba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:31:48.455370 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c22a694-09b7-4c0f-92a4-694484711c37 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:48.485182 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 366/714] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:31:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:31:48.485182 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74eb7d09-8f56-422e-97c8-4380b6df3cba None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:48.520668 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646a1fef-86fd-4f90-afc3-de3bddc47a36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:48.521953 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646a1fef-86fd-4f90-afc3-de3bddc47a36 None admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:48.522710 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646a1fef-86fd-4f90-afc3-de3bddc47a36 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:48.523205 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646a1fef-86fd-4f90-afc3-de3bddc47a36 None admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:48.524734 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646a1fef-86fd-4f90-afc3-de3bddc47a36 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:48.527144 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646a1fef-86fd-4f90-afc3-de3bddc47a36 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:48.557012 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 349/715] 10.222.0.89 () {60 vars in 1132 bytes} [Thu Jul 28 17:31:48 2022] POST /identity/v3/users => generated 254 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:31:48.558392 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646a1fef-86fd-4f90-afc3-de3bddc47a36 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:48.663556 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:48.677462 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74eb7d09-8f56-422e-97c8-4380b6df3cba None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:48.737224 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:48.756026 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6492ca50-2366-401e-b98c-acf7d7648074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:48.757007 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6492ca50-2366-401e-b98c-acf7d7648074 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:48.757542 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6492ca50-2366-401e-b98c-acf7d7648074 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:48.758034 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6492ca50-2366-401e-b98c-acf7d7648074 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:48.760407 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6492ca50-2366-401e-b98c-acf7d7648074 None admin] RBAC: Adding query filter params (role.id=73e457231a824ea98c28d22610be640c, scope.project.id=04434e8535a64044996316f1d17dc0bc, user.id=d10b49b29df94404a71f43273839c252) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:48.760979 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6492ca50-2366-401e-b98c-acf7d7648074 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:48.761304 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646a1fef-86fd-4f90-afc3-de3bddc47a36 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:48.762905 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6492ca50-2366-401e-b98c-acf7d7648074 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:48.770128 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 367/716] 10.222.0.89 () {56 vars in 1374 bytes} [Thu Jul 28 17:31:48 2022] GET /identity/v3/role_assignments?user.id=d10b49b29df94404a71f43273839c252&scope.project.id=04434e8535a64044996316f1d17dc0bc&role.id=73e457231a824ea98c28d22610be640c => generated 259 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:48.801973 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3232d55-afab-4efd-9b99-6c5f8e7d30b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:48.802636 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3232d55-afab-4efd-9b99-6c5f8e7d30b5 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:48.802914 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3232d55-afab-4efd-9b99-6c5f8e7d30b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:48.803191 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3232d55-afab-4efd-9b99-6c5f8e7d30b5 None admin] PATH_INFO: `/v3/services/network` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:48.803837 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3232d55-afab-4efd-9b99-6c5f8e7d30b5 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:48.805373 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3232d55-afab-4efd-9b99-6c5f8e7d30b5 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:48.819948 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-a3232d55-afab-4efd-9b99-6c5f8e7d30b5 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 17:31:48.819948 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:48.819948 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 17:31:48.819948 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 17:31:48.819948 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 17:31:48.819948 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 17:31:48.819948 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 17:31:48.819948 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:31:48.819948 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 17:31:48.819948 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:31:48.819948 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 17:31:48.819948 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:31:48.819948 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:48.819948 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:48.819948 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:48.819948 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:48.819948 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:48.820984 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:48.820984 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:48.820984 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:48.820984 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:48.820984 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:48.820984 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:48.820984 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 17:31:48.820984 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 17:31:48.820984 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 17:31:48.820984 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 17:31:48.820984 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:48.820984 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:48.820984 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:48.820984 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:48.820984 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:48.820984 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:48.821902 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:48.821902 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:48.821902 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:48.821902 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:48.821902 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:48.821902 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:48.821902 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:48.821902 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:48.821902 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:48.821902 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:48.821902 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 17:31:48.821902 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:31:48.821902 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 17:31:48.821902 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:31:48.822668 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 350/717] 10.222.0.89 () {56 vars in 1111 bytes} [Thu Jul 28 17:31:48 2022] GET /identity/v3/services/network => generated 88 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:31:48.829091 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6492ca50-2366-401e-b98c-acf7d7648074 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:48.947177 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6492ca50-2366-401e-b98c-acf7d7648074 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:48.990743 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbada558-52de-4841-bc71-3b5b616ac837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:48.991381 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbada558-52de-4841-bc71-3b5b616ac837 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:48.991661 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbada558-52de-4841-bc71-3b5b616ac837 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:48.991934 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbada558-52de-4841-bc71-3b5b616ac837 None admin] PATH_INFO: `/v3/services` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:48.992634 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbada558-52de-4841-bc71-3b5b616ac837 None admin] RBAC: Adding query filter params (name=network) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:48.992941 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbada558-52de-4841-bc71-3b5b616ac837 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:48.994405 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbada558-52de-4841-bc71-3b5b616ac837 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:49.001373 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 368/718] 10.222.0.89 () {56 vars in 1112 bytes} [Thu Jul 28 17:31:48 2022] GET /identity/v3/services?name=network => generated 124 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:49.006837 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3232d55-afab-4efd-9b99-6c5f8e7d30b5 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:49.120298 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3232d55-afab-4efd-9b99-6c5f8e7d30b5 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:49.172858 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4001dd6e-14cf-4f3c-9a0d-7ae8fe945631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:49.173747 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4001dd6e-14cf-4f3c-9a0d-7ae8fe945631 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:49.173938 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4001dd6e-14cf-4f3c-9a0d-7ae8fe945631 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:49.174169 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4001dd6e-14cf-4f3c-9a0d-7ae8fe945631 None admin] PATH_INFO: `/v3/services` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:49.174855 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4001dd6e-14cf-4f3c-9a0d-7ae8fe945631 None admin] RBAC: Adding query filter params (type=network) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:49.175156 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4001dd6e-14cf-4f3c-9a0d-7ae8fe945631 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:49.176523 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4001dd6e-14cf-4f3c-9a0d-7ae8fe945631 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:49.183345 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 351/719] 10.222.0.89 () {56 vars in 1112 bytes} [Thu Jul 28 17:31:49 2022] GET /identity/v3/services?type=network => generated 124 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:49.686816 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df516ca0-8c06-4d0c-b166-aeb23f9cd586 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:49.687370 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df516ca0-8c06-4d0c-b166-aeb23f9cd586 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:49.687370 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df516ca0-8c06-4d0c-b166-aeb23f9cd586 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:49.688906 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 369/720] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:31:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:31:49.694564 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d73680b-4ed0-4bab-8a77-72e95a2a53ef None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:49.694985 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d73680b-4ed0-4bab-8a77-72e95a2a53ef None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:49.695175 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d73680b-4ed0-4bab-8a77-72e95a2a53ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:49.722158 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-7d73680b-4ed0-4bab-8a77-72e95a2a53ef None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:31:49.723432 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-7d73680b-4ed0-4bab-8a77-72e95a2a53ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:31:49.731435 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:49.821102 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 352/721] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:31:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:31:49.863809 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df516ca0-8c06-4d0c-b166-aeb23f9cd586 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:49.901453 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:49.929092 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f29387d-5938-4feb-8c25-5f1a1652a333 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:49.929420 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f29387d-5938-4feb-8c25-5f1a1652a333 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:49.929674 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f29387d-5938-4feb-8c25-5f1a1652a333 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:49.931262 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 353/722] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:31:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:31:49.938921 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad4099a-aae8-45fe-96dc-d299f0cbb93f None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:49.939289 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad4099a-aae8-45fe-96dc-d299f0cbb93f None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:49.939788 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad4099a-aae8-45fe-96dc-d299f0cbb93f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:49.974144 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-9ad4099a-aae8-45fe-96dc-d299f0cbb93f None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:31:49.975600 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-9ad4099a-aae8-45fe-96dc-d299f0cbb93f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:31:50.031917 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df516ca0-8c06-4d0c-b166-aeb23f9cd586 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:50.083090 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b185730e-e321-4517-91a5-3d5a29e7baf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:50.084052 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b185730e-e321-4517-91a5-3d5a29e7baf9 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:50.084503 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b185730e-e321-4517-91a5-3d5a29e7baf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:50.084959 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b185730e-e321-4517-91a5-3d5a29e7baf9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:50.097494 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b185730e-e321-4517-91a5-3d5a29e7baf9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:31:50.103780 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-b185730e-e321-4517-91a5-3d5a29e7baf9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:31:50.103780 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:50.103780 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:50.103780 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:50.103780 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:50.103780 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:50.103780 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:50.103780 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:50.103780 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:50.103780 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:50.103780 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:50.103780 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:50.103780 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:31:50.103780 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:31:50.103780 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:31:50.103780 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:31:50.104707 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:31:50.104707 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:31:50.104707 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:31:50.104707 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:31:50.104707 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:50.104707 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:50.104707 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:50.104707 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:50.104707 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:50.104707 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:50.104707 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:50.104707 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:50.104707 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:50.104707 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:50.104707 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:50.104707 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:50.105586 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:50.105586 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:50.105586 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:50.105586 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:50.105586 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:31:50.105586 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:31:50.105586 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:31:50.105586 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:31:50.105586 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:31:50.105586 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:31:50.105586 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:31:50.105586 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:31:50.109404 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 370/723] 10.222.0.89 () {56 vars in 1096 bytes} [Thu Jul 28 17:31:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:31:50.118282 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b185730e-e321-4517-91a5-3d5a29e7baf9 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:50.118424 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 354/724] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:31:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 17:31:50.145834 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad4099a-aae8-45fe-96dc-d299f0cbb93f None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:50.250664 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:50.261662 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:50.287419 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad4099a-aae8-45fe-96dc-d299f0cbb93f None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:50.324804 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cedab3-7366-4a91-bbf8-b26a6b12d3d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:50.325475 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cedab3-7366-4a91-bbf8-b26a6b12d3d1 None admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:50.325752 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cedab3-7366-4a91-bbf8-b26a6b12d3d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:50.326023 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cedab3-7366-4a91-bbf8-b26a6b12d3d1 None admin] PATH_INFO: `/v3/services` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:50.327633 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b185730e-e321-4517-91a5-3d5a29e7baf9 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:50.327792 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cedab3-7366-4a91-bbf8-b26a6b12d3d1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:50.329437 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cedab3-7366-4a91-bbf8-b26a6b12d3d1 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:50.350328 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 355/725] 10.222.0.89 () {60 vars in 1141 bytes} [Thu Jul 28 17:31:50 2022] POST /identity/v3/services => generated 251 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 17:31:50.388194 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706d2e8e-494d-4d0a-9100-08fd844f25bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:50.389597 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706d2e8e-494d-4d0a-9100-08fd844f25bd None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:50.390317 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706d2e8e-494d-4d0a-9100-08fd844f25bd None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:50.390591 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706d2e8e-494d-4d0a-9100-08fd844f25bd None admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:50.391312 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706d2e8e-494d-4d0a-9100-08fd844f25bd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:50.391611 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706d2e8e-494d-4d0a-9100-08fd844f25bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:50.392993 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706d2e8e-494d-4d0a-9100-08fd844f25bd None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:50.403102 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 371/726] 10.222.0.89 () {56 vars in 1099 bytes} [Thu Jul 28 17:31:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:50.409141 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cedab3-7366-4a91-bbf8-b26a6b12d3d1 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:50.551418 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cedab3-7366-4a91-bbf8-b26a6b12d3d1 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:50.595690 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:50.602915 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba2951d-12ed-4f52-a841-79d2856758f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:50.603744 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba2951d-12ed-4f52-a841-79d2856758f6 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:50.604156 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba2951d-12ed-4f52-a841-79d2856758f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:50.604578 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba2951d-12ed-4f52-a841-79d2856758f6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:50.623251 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba2951d-12ed-4f52-a841-79d2856758f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:50.625169 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba2951d-12ed-4f52-a841-79d2856758f6 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:50.640678 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-fba2951d-12ed-4f52-a841-79d2856758f6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:31:50.640678 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:50.640678 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:50.640678 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:50.640678 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:50.640678 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:50.640678 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:50.640678 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:50.640678 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:50.640678 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:50.640678 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:50.640678 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:50.640678 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:31:50.640678 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:31:50.640678 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:31:50.641691 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:31:50.641691 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:50.641691 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:50.641691 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:50.641691 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:50.641691 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:50.641691 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:50.641691 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:50.641691 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:50.641691 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:50.641691 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:50.641691 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:50.641691 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:50.641691 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:50.641691 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:50.641691 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:50.641691 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:50.642614 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:31:50.642614 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:31:50.642614 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:31:50.642614 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:31:50.642614 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 356/727] 10.222.0.89 () {56 vars in 1108 bytes} [Thu Jul 28 17:31:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:31:50.647662 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706d2e8e-494d-4d0a-9100-08fd844f25bd None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:50.766366 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706d2e8e-494d-4d0a-9100-08fd844f25bd None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:50.807015 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d336be-eb5b-448c-bee5-e8cd4e6e4297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:50.807555 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d336be-eb5b-448c-bee5-e8cd4e6e4297 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:50.807831 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d336be-eb5b-448c-bee5-e8cd4e6e4297 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:50.808102 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d336be-eb5b-448c-bee5-e8cd4e6e4297 None admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:50.808802 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d336be-eb5b-448c-bee5-e8cd4e6e4297 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:50.809102 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d336be-eb5b-448c-bee5-e8cd4e6e4297 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:50.810654 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d336be-eb5b-448c-bee5-e8cd4e6e4297 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:50.826750 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 372/728] 10.222.0.89 () {56 vars in 1109 bytes} [Thu Jul 28 17:31:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:31:50.834896 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba2951d-12ed-4f52-a841-79d2856758f6 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:50.957382 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba2951d-12ed-4f52-a841-79d2856758f6 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:50.998364 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f34f89-8c3b-428b-aeba-50d64acd1d2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:50.999016 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f34f89-8c3b-428b-aeba-50d64acd1d2e None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:50.999297 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f34f89-8c3b-428b-aeba-50d64acd1d2e None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:50.999621 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f34f89-8c3b-428b-aeba-50d64acd1d2e None admin] PATH_INFO: `/v3/users/swift` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:51.007728 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f34f89-8c3b-428b-aeba-50d64acd1d2e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:51.009184 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f34f89-8c3b-428b-aeba-50d64acd1d2e None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:51.021800 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-39f34f89-8c3b-428b-aeba-50d64acd1d2e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 28 17:31:51.021800 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:51.021800 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:51.021800 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:51.021800 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:51.021800 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:51.021800 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:51.021800 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:51.021800 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:51.021800 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:51.021800 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:51.021800 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:51.021800 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 17:31:51.021800 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 17:31:51.021800 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 17:31:51.021800 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 17:31:51.022829 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:51.022829 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:51.022829 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 17:31:51.022829 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:31:51.022829 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 17:31:51.022829 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:31:51.022829 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:51.022829 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:51.022829 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:51.022829 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:51.022829 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:51.022829 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:51.022829 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:51.022829 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:51.022829 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:51.022829 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:51.022829 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:51.022829 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:51.023800 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:51.023800 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:51.023800 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 17:31:51.023800 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 17:31:51.023800 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 17:31:51.023800 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 17:31:51.023800 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 17:31:51.023800 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 17:31:51.023800 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 28 17:31:51.023800 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:31:51.023800 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 357/729] 10.222.0.89 () {56 vars in 1096 bytes} [Thu Jul 28 17:31:50 2022] GET /identity/v3/users/swift => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:31:51.034317 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d336be-eb5b-448c-bee5-e8cd4e6e4297 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:51.037344 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:51.197243 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d336be-eb5b-448c-bee5-e8cd4e6e4297 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:51.243791 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6735b908-b252-41ae-b9a9-29b8d3a095bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:51.244764 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6735b908-b252-41ae-b9a9-29b8d3a095bd None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:51.245559 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6735b908-b252-41ae-b9a9-29b8d3a095bd None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:51.246030 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6735b908-b252-41ae-b9a9-29b8d3a095bd None admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:51.247021 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6735b908-b252-41ae-b9a9-29b8d3a095bd None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:51.247527 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6735b908-b252-41ae-b9a9-29b8d3a095bd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:51.250292 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6735b908-b252-41ae-b9a9-29b8d3a095bd None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:51.278155 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 373/730] 10.222.0.89 () {56 vars in 1135 bytes} [Thu Jul 28 17:31:51 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:51.284145 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f34f89-8c3b-428b-aeba-50d64acd1d2e None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:51.358584 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:51.381759 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a884f940-d1c0-45a1-b8d2-8868edc85ee1 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:51.382581 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a884f940-d1c0-45a1-b8d2-8868edc85ee1 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:51.383066 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a884f940-d1c0-45a1-b8d2-8868edc85ee1 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:51.385026 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 374/731] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:31:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 17:31:51.391546 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5806a5c-c578-4e08-a127-659f513d22e3 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:51.392069 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5806a5c-c578-4e08-a127-659f513d22e3 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:51.392543 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5806a5c-c578-4e08-a127-659f513d22e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:51.415024 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-c5806a5c-c578-4e08-a127-659f513d22e3 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:31:51.416598 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-c5806a5c-c578-4e08-a127-659f513d22e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:31:51.437500 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f34f89-8c3b-428b-aeba-50d64acd1d2e None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:51.474857 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e84c12-b2a8-4b03-90c1-28715281c01c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:51.475402 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e84c12-b2a8-4b03-90c1-28715281c01c None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:51.475680 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e84c12-b2a8-4b03-90c1-28715281c01c None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:51.475953 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e84c12-b2a8-4b03-90c1-28715281c01c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:51.476487 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:51.489285 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 375/732] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:31:51 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:31:51.490907 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa59065-2514-4452-8e87-264abbf37053 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:51.491365 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa59065-2514-4452-8e87-264abbf37053 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:51.491832 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa59065-2514-4452-8e87-264abbf37053 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:51.493511 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 376/733] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:31:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:31:51.506548 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be070e27-08bc-4113-a769-25671ab99860 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:51.507143 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be070e27-08bc-4113-a769-25671ab99860 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:51.507611 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be070e27-08bc-4113-a769-25671ab99860 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:51.531506 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-be070e27-08bc-4113-a769-25671ab99860 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:31:51.532699 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-be070e27-08bc-4113-a769-25671ab99860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:31:51.556173 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e84c12-b2a8-4b03-90c1-28715281c01c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:51.557669 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e84c12-b2a8-4b03-90c1-28715281c01c None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:51.610494 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:51.617249 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 377/734] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:31:51 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:31:51.618132 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be070e27-08bc-4113-a769-25671ab99860 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:51.700922 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 358/735] 10.222.0.89 () {58 vars in 1282 bytes} [Thu Jul 28 17:31:51 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 17:31:51.702482 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e84c12-b2a8-4b03-90c1-28715281c01c None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:51.724732 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be070e27-08bc-4113-a769-25671ab99860 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:51.753199 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:51.768836 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731702bf-a9e9-4fec-9142-facc0a5693e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:51.770292 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731702bf-a9e9-4fec-9142-facc0a5693e6 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:51.770778 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731702bf-a9e9-4fec-9142-facc0a5693e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:51.771245 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731702bf-a9e9-4fec-9142-facc0a5693e6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:51.782440 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731702bf-a9e9-4fec-9142-facc0a5693e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:31:51.787557 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-731702bf-a9e9-4fec-9142-facc0a5693e6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 17:31:51.787557 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:51.787557 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:51.787557 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:51.787557 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:51.787557 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:51.787557 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:51.787557 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:51.787557 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:51.787557 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:51.787557 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:51.787557 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:51.787557 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:31:51.787557 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:31:51.787557 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:31:51.787557 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:31:51.788493 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:31:51.788493 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:31:51.788493 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:31:51.788493 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:31:51.788493 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:51.788493 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:51.788493 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:51.788493 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:51.788493 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:51.788493 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:51.788493 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:51.788493 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:51.788493 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:51.788493 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:51.788493 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:51.788493 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:51.789402 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:51.789402 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:51.789402 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:51.789402 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:51.789402 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:31:51.789402 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:31:51.789402 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:31:51.789402 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:31:51.789402 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:31:51.789402 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:31:51.789402 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 17:31:51.789402 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:31:51.792279 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 378/736] 10.222.0.89 () {56 vars in 1102 bytes} [Thu Jul 28 17:31:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:31:51.793524 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731702bf-a9e9-4fec-9142-facc0a5693e6 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:51.836412 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e84c12-b2a8-4b03-90c1-28715281c01c None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:51.881366 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39f0ef9-f726-4687-9fdb-a8c55bfb0f86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:51.882017 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39f0ef9-f726-4687-9fdb-a8c55bfb0f86 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:51.882297 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39f0ef9-f726-4687-9fdb-a8c55bfb0f86 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:51.882572 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39f0ef9-f726-4687-9fdb-a8c55bfb0f86 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:51.896492 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39f0ef9-f726-4687-9fdb-a8c55bfb0f86 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:31:51.903131 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-d39f0ef9-f726-4687-9fdb-a8c55bfb0f86 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:31:51.903131 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:51.903131 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:51.903131 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:51.903131 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:51.903131 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:51.903131 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:51.903131 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:51.903131 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:51.903131 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:51.903131 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:51.903131 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:51.903131 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:31:51.903131 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:31:51.903131 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:31:51.903131 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:31:51.904107 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:31:51.904107 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:31:51.904107 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:31:51.904107 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:31:51.904107 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:51.904107 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:51.904107 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:51.904107 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:51.904107 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:51.904107 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:51.904107 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:51.904107 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:51.904107 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:51.904107 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:51.904107 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:51.904107 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:51.905817 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:51.905817 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:51.905817 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:51.905817 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:51.905817 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:31:51.905817 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:31:51.905817 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:31:51.905817 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:31:51.905817 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:31:51.905817 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:31:51.905817 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:31:51.905817 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:31:51.909232 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 359/737] 10.222.0.89 () {56 vars in 1096 bytes} [Thu Jul 28 17:31:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:31:51.910410 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39f0ef9-f726-4687-9fdb-a8c55bfb0f86 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:51.913819 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731702bf-a9e9-4fec-9142-facc0a5693e6 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:51.915920 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:51.954996 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3135c844-7b7e-418b-9c56-d240c91233d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:51.955746 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3135c844-7b7e-418b-9c56-d240c91233d3 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:51.956022 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3135c844-7b7e-418b-9c56-d240c91233d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:51.957047 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3135c844-7b7e-418b-9c56-d240c91233d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:51.968500 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3135c844-7b7e-418b-9c56-d240c91233d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:51.970197 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3135c844-7b7e-418b-9c56-d240c91233d3 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:51.985853 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-3135c844-7b7e-418b-9c56-d240c91233d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:31:51.985853 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:51.985853 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:51.985853 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:51.985853 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:51.985853 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:51.985853 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:51.985853 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:51.985853 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:51.985853 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:51.985853 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:51.985853 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:51.985853 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:31:51.985853 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:31:51.985853 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:31:51.986799 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:31:51.986799 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:51.986799 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:51.986799 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:51.986799 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:51.986799 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:51.986799 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:51.986799 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:51.986799 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:51.986799 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:51.986799 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:51.986799 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:51.986799 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:51.986799 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:51.986799 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:51.986799 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:51.986799 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:51.987682 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:31:51.987682 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:31:51.987682 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:31:51.987682 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:31:51.987682 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 379/738] 10.222.0.89 () {56 vars in 1108 bytes} [Thu Jul 28 17:31:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:31:51.987964 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3135c844-7b7e-418b-9c56-d240c91233d3 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:52.030028 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39f0ef9-f726-4687-9fdb-a8c55bfb0f86 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:52.032084 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:52.067558 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a8e904-d729-42d4-b53b-6743d1ac16a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:52.068174 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a8e904-d729-42d4-b53b-6743d1ac16a0 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:52.068458 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a8e904-d729-42d4-b53b-6743d1ac16a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:52.068728 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a8e904-d729-42d4-b53b-6743d1ac16a0 None admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:52.069475 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a8e904-d729-42d4-b53b-6743d1ac16a0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:52.069783 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a8e904-d729-42d4-b53b-6743d1ac16a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:52.071132 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a8e904-d729-42d4-b53b-6743d1ac16a0 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:52.080227 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 360/739] 10.222.0.89 () {56 vars in 1103 bytes} [Thu Jul 28 17:31:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:52.081383 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b78d243-cd8c-488f-ab97-afd00d8b22c8 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:52.081669 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b78d243-cd8c-488f-ab97-afd00d8b22c8 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:52.081939 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b78d243-cd8c-488f-ab97-afd00d8b22c8 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:52.087353 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 361/740] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:31:52 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:31:52.087353 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b78d243-cd8c-488f-ab97-afd00d8b22c8 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:52.129343 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3135c844-7b7e-418b-9c56-d240c91233d3 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:52.157930 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:52.168602 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d1cea0-ad35-4d55-983f-6345a51b1809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:52.169666 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d1cea0-ad35-4d55-983f-6345a51b1809 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:52.169985 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d1cea0-ad35-4d55-983f-6345a51b1809 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:52.170265 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d1cea0-ad35-4d55-983f-6345a51b1809 None admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:52.170989 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d1cea0-ad35-4d55-983f-6345a51b1809 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:52.171294 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d1cea0-ad35-4d55-983f-6345a51b1809 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:52.173391 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d1cea0-ad35-4d55-983f-6345a51b1809 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:52.182925 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 380/741] 10.222.0.89 () {56 vars in 1099 bytes} [Thu Jul 28 17:31:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:52.183409 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d1cea0-ad35-4d55-983f-6345a51b1809 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:52.201041 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b78d243-cd8c-488f-ab97-afd00d8b22c8 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:52.240498 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115d8b23-b178-4bad-b3af-c1ea586db9e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:52.241613 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115d8b23-b178-4bad-b3af-c1ea586db9e2 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:52.241956 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115d8b23-b178-4bad-b3af-c1ea586db9e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:52.242449 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115d8b23-b178-4bad-b3af-c1ea586db9e2 None admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:52.243208 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115d8b23-b178-4bad-b3af-c1ea586db9e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:52.243680 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115d8b23-b178-4bad-b3af-c1ea586db9e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:52.245684 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115d8b23-b178-4bad-b3af-c1ea586db9e2 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:52.261510 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 362/742] 10.222.0.89 () {56 vars in 1109 bytes} [Thu Jul 28 17:31:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:52.262688 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6c420c-fc59-445e-a70a-8c76a84bec35 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:52.262998 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6c420c-fc59-445e-a70a-8c76a84bec35 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:52.263294 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6c420c-fc59-445e-a70a-8c76a84bec35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:52.283373 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-1f6c420c-fc59-445e-a70a-8c76a84bec35 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:31:52.284577 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-1f6c420c-fc59-445e-a70a-8c76a84bec35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:31:52.292861 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:52.301018 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d1cea0-ad35-4d55-983f-6345a51b1809 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:52.335647 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb76f56-0142-4034-b0df-a7a8f15f261b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:52.336279 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb76f56-0142-4034-b0df-a7a8f15f261b None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:52.336558 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb76f56-0142-4034-b0df-a7a8f15f261b None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:52.336828 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb76f56-0142-4034-b0df-a7a8f15f261b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:52.348377 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb76f56-0142-4034-b0df-a7a8f15f261b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:52.349848 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb76f56-0142-4034-b0df-a7a8f15f261b None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:52.369205 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 363/743] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:31:52 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 17:31:52.369205 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6c420c-fc59-445e-a70a-8c76a84bec35 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:52.369476 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-2fb76f56-0142-4034-b0df-a7a8f15f261b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:31:52.369476 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:52.369476 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:52.369476 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:52.369476 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:52.369476 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:52.369476 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:52.369476 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:52.369476 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:52.369476 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:52.369476 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:52.369476 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:52.369476 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:31:52.369476 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:31:52.369476 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:31:52.370551 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:31:52.370551 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:52.370551 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:52.370551 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:52.370551 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:52.370551 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:52.370551 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:52.370551 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:52.370551 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:52.370551 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:52.370551 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:52.370551 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:52.370551 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:52.370551 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:52.370551 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:52.370551 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:52.370551 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:52.371459 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:31:52.371459 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:31:52.371459 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:31:52.371459 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:31:52.371459 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 381/744] 10.222.0.89 () {56 vars in 1108 bytes} [Thu Jul 28 17:31:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:31:52.371459 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb76f56-0142-4034-b0df-a7a8f15f261b None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:52.431426 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:52.478593 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6c420c-fc59-445e-a70a-8c76a84bec35 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:52.478809 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb76f56-0142-4034-b0df-a7a8f15f261b None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:52.513412 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f488f7c-6ace-4eb0-ab87-1ba72e238798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:52.514610 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f488f7c-6ace-4eb0-ab87-1ba72e238798 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:52.514901 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f488f7c-6ace-4eb0-ab87-1ba72e238798 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:52.515152 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f488f7c-6ace-4eb0-ab87-1ba72e238798 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:52.516611 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba4265f-5864-4f29-b4bc-b872c6fa378e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:52.517277 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba4265f-5864-4f29-b4bc-b872c6fa378e None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:52.517560 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba4265f-5864-4f29-b4bc-b872c6fa378e None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:52.517826 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba4265f-5864-4f29-b4bc-b872c6fa378e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:52.524718 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba4265f-5864-4f29-b4bc-b872c6fa378e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:52.525947 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f488f7c-6ace-4eb0-ab87-1ba72e238798 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:52.526191 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba4265f-5864-4f29-b4bc-b872c6fa378e None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:52.527315 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f488f7c-6ace-4eb0-ab87-1ba72e238798 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:52.538441 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-1ba4265f-5864-4f29-b4bc-b872c6fa378e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:31:52.538441 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:52.538441 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:52.538441 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:52.538441 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:52.538441 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:52.538441 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:52.538441 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:52.538441 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:52.538441 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:52.538441 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:52.538441 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:52.538441 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 17:31:52.538441 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 17:31:52.538441 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 17:31:52.539338 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 17:31:52.539338 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:52.539338 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:52.539338 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:52.539338 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:52.539338 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:52.539338 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:52.539338 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:52.539338 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:52.539338 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:52.539338 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:52.539338 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:52.539338 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:52.539338 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:52.539338 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:52.539338 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:52.539338 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:52.540314 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 17:31:52.540314 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 17:31:52.540314 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 17:31:52.540314 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 17:31:52.540314 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 17:31:52.540314 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 17:31:52.540314 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:31:52.540314 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:31:52.540314 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 382/745] 10.222.0.89 () {56 vars in 1111 bytes} [Thu Jul 28 17:31:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:31:52.540800 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba4265f-5864-4f29-b4bc-b872c6fa378e None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:52.547853 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-2f488f7c-6ace-4eb0-ab87-1ba72e238798 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:31:52.547853 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:52.547853 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:52.547853 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:52.547853 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:52.547853 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:52.547853 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:52.547853 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:52.547853 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:52.547853 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:52.547853 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:52.547853 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:52.547853 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:31:52.547853 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:31:52.547853 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:31:52.548760 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:31:52.548760 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:52.548760 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:52.548760 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:52.548760 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:52.548760 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:52.548760 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:52.548760 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:52.548760 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:52.548760 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:52.548760 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:52.548760 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:52.548760 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:52.548760 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:52.548760 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:52.548760 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:52.548760 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:52.549993 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:31:52.549993 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:31:52.549993 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:31:52.549993 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:31:52.550607 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:52.553341 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 364/746] 10.222.0.89 () {56 vars in 1108 bytes} [Thu Jul 28 17:31:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:31:52.554191 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f488f7c-6ace-4eb0-ab87-1ba72e238798 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:52.649612 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba4265f-5864-4f29-b4bc-b872c6fa378e None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:52.666762 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f488f7c-6ace-4eb0-ab87-1ba72e238798 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:52.676895 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:52.688317 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44361bf8-861d-49b7-98e0-67c41942cf10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:52.689220 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44361bf8-861d-49b7-98e0-67c41942cf10 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:52.689679 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44361bf8-861d-49b7-98e0-67c41942cf10 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:52.690560 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44361bf8-861d-49b7-98e0-67c41942cf10 None admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:52.691703 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44361bf8-861d-49b7-98e0-67c41942cf10 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:52.692211 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44361bf8-861d-49b7-98e0-67c41942cf10 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:52.694434 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44361bf8-861d-49b7-98e0-67c41942cf10 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:52.705314 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a67707-a59d-49ec-b0b2-5d7af00829eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:52.705944 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a67707-a59d-49ec-b0b2-5d7af00829eb None admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:52.706251 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a67707-a59d-49ec-b0b2-5d7af00829eb None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:52.706527 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a67707-a59d-49ec-b0b2-5d7af00829eb None admin] PATH_INFO: `/v3/services` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:52.707224 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a67707-a59d-49ec-b0b2-5d7af00829eb None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:52.708589 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a67707-a59d-49ec-b0b2-5d7af00829eb None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:52.710588 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 383/747] 10.222.0.89 () {56 vars in 1109 bytes} [Thu Jul 28 17:31:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:52.711652 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44361bf8-861d-49b7-98e0-67c41942cf10 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:52.733193 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 365/748] 10.222.0.89 () {60 vars in 1141 bytes} [Thu Jul 28 17:31:52 2022] POST /identity/v3/services => generated 240 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:31:52.733193 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a67707-a59d-49ec-b0b2-5d7af00829eb None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:52.798136 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:52.852615 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44361bf8-861d-49b7-98e0-67c41942cf10 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:52.901089 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a67707-a59d-49ec-b0b2-5d7af00829eb None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:52.907738 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556a35ef-644e-42d9-be9a-092869adb3ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:52.908352 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556a35ef-644e-42d9-be9a-092869adb3ea None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:52.908635 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556a35ef-644e-42d9-be9a-092869adb3ea None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:52.909347 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556a35ef-644e-42d9-be9a-092869adb3ea None admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:52.909698 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556a35ef-644e-42d9-be9a-092869adb3ea None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:52.910002 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556a35ef-644e-42d9-be9a-092869adb3ea None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:52.911957 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556a35ef-644e-42d9-be9a-092869adb3ea None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:52.926460 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 384/749] 10.222.0.89 () {56 vars in 1148 bytes} [Thu Jul 28 17:31:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:52.926929 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556a35ef-644e-42d9-be9a-092869adb3ea None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:52.948760 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42d60ea-32a6-41d2-a314-c44912588e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:52.949415 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42d60ea-32a6-41d2-a314-c44912588e06 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:52.949709 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42d60ea-32a6-41d2-a314-c44912588e06 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:52.949976 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42d60ea-32a6-41d2-a314-c44912588e06 None admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:52.950679 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42d60ea-32a6-41d2-a314-c44912588e06 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:52.950985 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42d60ea-32a6-41d2-a314-c44912588e06 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:52.952450 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42d60ea-32a6-41d2-a314-c44912588e06 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:52.966449 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 366/750] 10.222.0.89 () {56 vars in 1109 bytes} [Thu Jul 28 17:31:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:52.967972 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42d60ea-32a6-41d2-a314-c44912588e06 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:52.997439 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:53.046902 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556a35ef-644e-42d9-be9a-092869adb3ea None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:53.071709 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:53.082345 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42d60ea-32a6-41d2-a314-c44912588e06 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:53.087493 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4e451f-174a-46f4-b3a2-6bd4a401eb0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:53.088123 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4e451f-174a-46f4-b3a2-6bd4a401eb0b None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:53.088397 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4e451f-174a-46f4-b3a2-6bd4a401eb0b None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:53.088664 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4e451f-174a-46f4-b3a2-6bd4a401eb0b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:53.097102 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4e451f-174a-46f4-b3a2-6bd4a401eb0b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:53.098564 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4e451f-174a-46f4-b3a2-6bd4a401eb0b None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:53.112465 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-2c4e451f-174a-46f4-b3a2-6bd4a401eb0b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 17:31:53.112465 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:53.112465 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:53.112465 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:53.112465 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:53.112465 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:53.112465 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:53.112465 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:53.112465 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:53.112465 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:53.112465 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:53.112465 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:53.112465 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 17:31:53.112465 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 17:31:53.112465 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 17:31:53.112465 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 17:31:53.113762 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:53.113762 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:53.113762 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 17:31:53.113762 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:31:53.113762 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 17:31:53.113762 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:31:53.113762 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:53.113762 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:53.113762 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:53.113762 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:53.113762 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:53.113762 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:53.113762 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:53.113762 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:53.113762 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:53.113762 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:53.113762 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:53.113762 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:53.114724 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:53.114724 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:53.114724 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 17:31:53.114724 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 17:31:53.114724 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 17:31:53.114724 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 17:31:53.114724 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 17:31:53.114724 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 17:31:53.114724 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 17:31:53.114724 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:31:53.114724 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 385/751] 10.222.0.89 () {56 vars in 1117 bytes} [Thu Jul 28 17:31:52 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:31:53.115895 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5348fc9b-1aac-48e6-983d-c8bc15536d32 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:53.116179 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5348fc9b-1aac-48e6-983d-c8bc15536d32 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:53.116445 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5348fc9b-1aac-48e6-983d-c8bc15536d32 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:53.119336 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 386/752] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:31:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:31:53.120349 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5348fc9b-1aac-48e6-983d-c8bc15536d32 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:53.124353 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fab89bc-0827-481a-b4e4-58681b195085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:53.125929 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fab89bc-0827-481a-b4e4-58681b195085 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:53.126223 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fab89bc-0827-481a-b4e4-58681b195085 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:53.126501 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fab89bc-0827-481a-b4e4-58681b195085 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:53.127323 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fab89bc-0827-481a-b4e4-58681b195085 None admin] RBAC: Adding query filter params (role.id=73e457231a824ea98c28d22610be640c, scope.project.id=04434e8535a64044996316f1d17dc0bc, user.id=3c59473b01bf4433ba3b111114fc048b) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:53.127637 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fab89bc-0827-481a-b4e4-58681b195085 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:53.129108 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fab89bc-0827-481a-b4e4-58681b195085 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:53.145208 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 367/753] 10.222.0.89 () {56 vars in 1374 bytes} [Thu Jul 28 17:31:52 2022] GET /identity/v3/role_assignments?user.id=3c59473b01bf4433ba3b111114fc048b&scope.project.id=04434e8535a64044996316f1d17dc0bc&role.id=73e457231a824ea98c28d22610be640c => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:53.145208 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fab89bc-0827-481a-b4e4-58681b195085 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:53.257784 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5348fc9b-1aac-48e6-983d-c8bc15536d32 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:53.282929 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:53.300439 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaa2ecb-e6af-4952-8c11-1c653bd8580e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:53.301265 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaa2ecb-e6af-4952-8c11-1c653bd8580e None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:53.301621 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaa2ecb-e6af-4952-8c11-1c653bd8580e None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:53.301824 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fab89bc-0827-481a-b4e4-58681b195085 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:53.302123 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaa2ecb-e6af-4952-8c11-1c653bd8580e None admin] PATH_INFO: `/v3/users/nova` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:53.315270 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcaa2ecb-e6af-4952-8c11-1c653bd8580e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:53.317352 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcaa2ecb-e6af-4952-8c11-1c653bd8580e None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:53.330789 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-fcaa2ecb-e6af-4952-8c11-1c653bd8580e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 17:31:53.330789 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:53.330789 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:53.330789 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:53.330789 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:53.330789 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:53.330789 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:53.330789 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:53.330789 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:53.330789 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:53.330789 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:53.330789 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:53.330789 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 17:31:53.330789 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 17:31:53.330789 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 17:31:53.330789 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 17:31:53.331792 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:53.331792 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:53.331792 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 17:31:53.331792 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:31:53.331792 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 17:31:53.331792 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:31:53.331792 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:53.331792 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:53.331792 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:53.331792 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:53.331792 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:53.331792 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:53.331792 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:53.331792 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:53.331792 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:53.331792 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:53.331792 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:53.331792 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:53.332778 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:53.332778 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:53.332778 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 17:31:53.332778 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 17:31:53.332778 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 17:31:53.332778 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 17:31:53.332778 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 17:31:53.332778 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 17:31:53.332778 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 17:31:53.332778 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:31:53.332778 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 387/754] 10.222.0.89 () {56 vars in 1093 bytes} [Thu Jul 28 17:31:53 2022] GET /identity/v3/users/nova => generated 82 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:31:53.333405 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab07959-8a71-4089-8fc2-b880ee2604dc None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:53.333482 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab07959-8a71-4089-8fc2-b880ee2604dc None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:53.334625 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab07959-8a71-4089-8fc2-b880ee2604dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:53.353511 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307820fd-cf80-452a-bbd2-7f28bc66fa1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:53.354136 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307820fd-cf80-452a-bbd2-7f28bc66fa1a None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:53.354416 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307820fd-cf80-452a-bbd2-7f28bc66fa1a None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:53.354689 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307820fd-cf80-452a-bbd2-7f28bc66fa1a None admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:53.355486 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307820fd-cf80-452a-bbd2-7f28bc66fa1a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:53.355793 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307820fd-cf80-452a-bbd2-7f28bc66fa1a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:53.357243 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307820fd-cf80-452a-bbd2-7f28bc66fa1a None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:53.363051 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-2ab07959-8a71-4089-8fc2-b880ee2604dc None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:31:53.364087 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-2ab07959-8a71-4089-8fc2-b880ee2604dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:31:53.383300 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 368/755] 10.222.0.89 () {56 vars in 1149 bytes} [Thu Jul 28 17:31:53 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:53.384510 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307820fd-cf80-452a-bbd2-7f28bc66fa1a None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:53.412199 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:53.457182 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 388/756] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:31:53 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:31:53.457182 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab07959-8a71-4089-8fc2-b880ee2604dc None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:53.539824 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307820fd-cf80-452a-bbd2-7f28bc66fa1a None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:53.581984 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1355577b-c0c0-45d8-a8d0-1bb39096067e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:53.582533 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1355577b-c0c0-45d8-a8d0-1bb39096067e None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:53.582814 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1355577b-c0c0-45d8-a8d0-1bb39096067e None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:53.583089 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1355577b-c0c0-45d8-a8d0-1bb39096067e None admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:53.583903 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1355577b-c0c0-45d8-a8d0-1bb39096067e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:53.584212 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1355577b-c0c0-45d8-a8d0-1bb39096067e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:53.585704 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1355577b-c0c0-45d8-a8d0-1bb39096067e None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:53.606305 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 369/757] 10.222.0.89 () {56 vars in 1133 bytes} [Thu Jul 28 17:31:53 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:53.611861 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1355577b-c0c0-45d8-a8d0-1bb39096067e None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:53.615939 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:53.708793 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab07959-8a71-4089-8fc2-b880ee2604dc None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:53.728365 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1355577b-c0c0-45d8-a8d0-1bb39096067e None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:53.746403 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92041f0-972f-4b2b-96cd-d07ea1484b50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:53.747351 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92041f0-972f-4b2b-96cd-d07ea1484b50 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:53.747824 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92041f0-972f-4b2b-96cd-d07ea1484b50 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:53.749545 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92041f0-972f-4b2b-96cd-d07ea1484b50 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:53.767425 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5c61fc-2bac-421a-9582-165c5b54cfd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:53.768124 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5c61fc-2bac-421a-9582-165c5b54cfd8 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:53.768346 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5c61fc-2bac-421a-9582-165c5b54cfd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:53.768622 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5c61fc-2bac-421a-9582-165c5b54cfd8 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:53.769361 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5c61fc-2bac-421a-9582-165c5b54cfd8 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:53.770829 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5c61fc-2bac-421a-9582-165c5b54cfd8 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:53.781365 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-4b5c61fc-2bac-421a-9582-165c5b54cfd8 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 17:31:53.781365 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:53.781365 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 17:31:53.781365 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 17:31:53.781365 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 17:31:53.781365 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 17:31:53.781365 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 17:31:53.781365 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:31:53.781365 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 17:31:53.781365 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:31:53.781365 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 17:31:53.781365 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:31:53.781365 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:53.781365 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:53.781365 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:53.781365 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:53.782321 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:53.782321 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:53.782321 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:53.782321 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:53.782321 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:53.782321 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:53.782321 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:53.782321 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 17:31:53.782321 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 17:31:53.782321 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 17:31:53.782321 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 17:31:53.782321 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:53.782321 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:53.782321 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:53.782321 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:53.782321 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:53.783173 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:53.783173 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:53.783173 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:53.783173 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:53.783173 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:53.783173 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:53.783173 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:53.783173 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:53.783173 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:53.783173 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:53.783173 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:53.783173 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 17:31:53.783173 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:31:53.783173 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 17:31:53.783173 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:31:53.787557 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 370/758] 10.222.0.89 () {56 vars in 1129 bytes} [Thu Jul 28 17:31:53 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:31:53.788390 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5c61fc-2bac-421a-9582-165c5b54cfd8 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:53.800526 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:53.831736 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92041f0-972f-4b2b-96cd-d07ea1484b50 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:53.833392 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92041f0-972f-4b2b-96cd-d07ea1484b50 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:53.877389 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:53.918237 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5c61fc-2bac-421a-9582-165c5b54cfd8 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:53.947067 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 389/759] 10.222.0.89 () {58 vars in 1282 bytes} [Thu Jul 28 17:31:53 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 492 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 17:31:53.947067 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92041f0-972f-4b2b-96cd-d07ea1484b50 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:53.966265 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733063ac-f004-4d6c-941d-179fd2f28c77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:53.967760 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733063ac-f004-4d6c-941d-179fd2f28c77 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:53.968227 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733063ac-f004-4d6c-941d-179fd2f28c77 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:53.968659 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733063ac-f004-4d6c-941d-179fd2f28c77 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:53.981771 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733063ac-f004-4d6c-941d-179fd2f28c77 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:53.984508 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733063ac-f004-4d6c-941d-179fd2f28c77 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:54.001543 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-733063ac-f004-4d6c-941d-179fd2f28c77 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:31:54.001543 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:54.001543 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:54.001543 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:54.001543 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:54.001543 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:54.001543 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:54.001543 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:54.001543 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:54.001543 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:54.001543 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:54.001543 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:54.001543 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:31:54.001543 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:31:54.001543 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:31:54.002686 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:31:54.002686 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:54.002686 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:54.002686 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:54.002686 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:54.002686 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:54.002686 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:54.002686 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:54.002686 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:54.002686 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:54.002686 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:54.002686 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:54.002686 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:54.002686 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:54.002686 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:54.002686 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:54.002686 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:54.003784 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:31:54.003784 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:31:54.003784 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:31:54.003784 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:31:54.010100 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 371/760] 10.222.0.89 () {56 vars in 1108 bytes} [Thu Jul 28 17:31:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:31:54.011251 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733063ac-f004-4d6c-941d-179fd2f28c77 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:54.024058 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:54.057869 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92041f0-972f-4b2b-96cd-d07ea1484b50 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:54.094015 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e6ff25-86bf-40df-b951-0f978f655b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:54.095193 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e6ff25-86bf-40df-b951-0f978f655b47 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:54.095193 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e6ff25-86bf-40df-b951-0f978f655b47 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:54.095193 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e6ff25-86bf-40df-b951-0f978f655b47 None admin] PATH_INFO: `/v3/services` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:54.096118 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e6ff25-86bf-40df-b951-0f978f655b47 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:54.096514 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e6ff25-86bf-40df-b951-0f978f655b47 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:54.097989 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e6ff25-86bf-40df-b951-0f978f655b47 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:54.107846 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 390/761] 10.222.0.89 () {56 vars in 1124 bytes} [Thu Jul 28 17:31:53 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:54.107846 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e6ff25-86bf-40df-b951-0f978f655b47 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:54.135322 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733063ac-f004-4d6c-941d-179fd2f28c77 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:54.157763 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:54.175026 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc71559-f984-493b-9b73-d11c3ce80c5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:54.176039 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc71559-f984-493b-9b73-d11c3ce80c5e None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:54.176524 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc71559-f984-493b-9b73-d11c3ce80c5e None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:54.177049 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc71559-f984-493b-9b73-d11c3ce80c5e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:54.190425 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc71559-f984-493b-9b73-d11c3ce80c5e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:54.192380 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc71559-f984-493b-9b73-d11c3ce80c5e None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:54.209389 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-edc71559-f984-493b-9b73-d11c3ce80c5e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:31:54.209389 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:54.209389 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:54.209389 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:54.209389 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:54.209389 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:54.209389 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:54.209389 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:54.209389 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:54.209389 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:54.209389 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:54.209389 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:54.209389 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:31:54.209389 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:31:54.209389 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:31:54.210357 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:31:54.210357 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:54.210357 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:54.210357 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:54.210357 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:54.210357 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:54.210357 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:54.210357 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:54.210357 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:54.210357 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:54.210357 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:54.210357 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:54.210357 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:54.210357 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:54.210357 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:54.210357 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:54.210357 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:54.211243 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:31:54.211243 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:31:54.211243 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:31:54.211243 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:31:54.211477 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e6ff25-86bf-40df-b951-0f978f655b47 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:54.218999 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 372/762] 10.222.0.89 () {56 vars in 1108 bytes} [Thu Jul 28 17:31:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:31:54.220260 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc71559-f984-493b-9b73-d11c3ce80c5e None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:54.251734 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6041adef-2c85-4462-8e9a-b6a805bc4e9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:54.252414 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6041adef-2c85-4462-8e9a-b6a805bc4e9e None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:54.252671 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6041adef-2c85-4462-8e9a-b6a805bc4e9e None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:54.252944 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6041adef-2c85-4462-8e9a-b6a805bc4e9e None admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:54.253865 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6041adef-2c85-4462-8e9a-b6a805bc4e9e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:54.254048 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6041adef-2c85-4462-8e9a-b6a805bc4e9e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:54.255539 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6041adef-2c85-4462-8e9a-b6a805bc4e9e None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:54.258145 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:54.276933 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 391/763] 10.222.0.89 () {56 vars in 1109 bytes} [Thu Jul 28 17:31:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:54.276933 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6041adef-2c85-4462-8e9a-b6a805bc4e9e None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:54.352781 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc71559-f984-493b-9b73-d11c3ce80c5e None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:54.392512 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6041adef-2c85-4462-8e9a-b6a805bc4e9e None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:54.397852 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897a32b4-655e-4350-97e2-215a7d0084bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:54.398831 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897a32b4-655e-4350-97e2-215a7d0084bc None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:54.399382 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897a32b4-655e-4350-97e2-215a7d0084bc None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:54.399862 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897a32b4-655e-4350-97e2-215a7d0084bc None admin] PATH_INFO: `/v3/services` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:54.400918 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897a32b4-655e-4350-97e2-215a7d0084bc None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:54.401503 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897a32b4-655e-4350-97e2-215a7d0084bc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:54.403102 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897a32b4-655e-4350-97e2-215a7d0084bc None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:54.409388 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 373/764] 10.222.0.89 () {56 vars in 1124 bytes} [Thu Jul 28 17:31:54 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:54.410689 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897a32b4-655e-4350-97e2-215a7d0084bc None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:54.437420 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f6a9e3-04b8-4aef-a58a-190552dfb206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:54.437846 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f6a9e3-04b8-4aef-a58a-190552dfb206 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:54.438122 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f6a9e3-04b8-4aef-a58a-190552dfb206 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:54.438394 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f6a9e3-04b8-4aef-a58a-190552dfb206 None admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:54.439084 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f6a9e3-04b8-4aef-a58a-190552dfb206 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:54.439384 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f6a9e3-04b8-4aef-a58a-190552dfb206 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:54.441135 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f6a9e3-04b8-4aef-a58a-190552dfb206 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:54.454303 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:54.461515 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 392/765] 10.222.0.89 () {56 vars in 1109 bytes} [Thu Jul 28 17:31:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:54.461515 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f6a9e3-04b8-4aef-a58a-190552dfb206 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:54.524737 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:54.539018 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897a32b4-655e-4350-97e2-215a7d0084bc None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:54.580428 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a245d7-8257-40ab-acdf-7c694f6796eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:54.581534 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a245d7-8257-40ab-acdf-7c694f6796eb None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:54.581648 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f6a9e3-04b8-4aef-a58a-190552dfb206 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:54.582133 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a245d7-8257-40ab-acdf-7c694f6796eb None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:54.582614 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a245d7-8257-40ab-acdf-7c694f6796eb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:54.592538 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a245d7-8257-40ab-acdf-7c694f6796eb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:54.594665 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a245d7-8257-40ab-acdf-7c694f6796eb None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:54.608353 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-88a245d7-8257-40ab-acdf-7c694f6796eb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:31:54.608353 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:54.608353 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:54.608353 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:54.608353 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:54.608353 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:54.608353 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:54.608353 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:54.608353 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:54.608353 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:54.608353 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:54.608353 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:54.608353 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 17:31:54.608353 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 17:31:54.608353 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 17:31:54.609607 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 17:31:54.609607 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:54.609607 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:54.609607 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:54.609607 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:54.609607 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:54.609607 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:54.609607 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:54.609607 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:54.609607 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:54.609607 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:54.609607 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:54.609607 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:54.609607 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:54.609607 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:54.609607 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:54.609607 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:54.610500 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 17:31:54.610500 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 17:31:54.610500 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 17:31:54.610500 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 17:31:54.610500 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 17:31:54.610500 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 17:31:54.610500 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:31:54.610500 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:31:54.617636 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 374/766] 10.222.0.89 () {56 vars in 1111 bytes} [Thu Jul 28 17:31:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:31:54.619006 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a245d7-8257-40ab-acdf-7c694f6796eb None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:54.624739 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664df705-e6ac-4d33-9932-74f558203810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:54.625339 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664df705-e6ac-4d33-9932-74f558203810 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:54.625617 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664df705-e6ac-4d33-9932-74f558203810 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:54.625887 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664df705-e6ac-4d33-9932-74f558203810 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:54.626633 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664df705-e6ac-4d33-9932-74f558203810 None admin] RBAC: Adding query filter params (interface=public, service_id=9266c70928fd4193b5702478615a2aec, region_id=RegionOne) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:54.626937 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664df705-e6ac-4d33-9932-74f558203810 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:54.628300 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664df705-e6ac-4d33-9932-74f558203810 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:54.636767 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 393/767] 10.222.0.89 () {56 vars in 1251 bytes} [Thu Jul 28 17:31:54 2022] GET /identity/v3/endpoints?service_id=9266c70928fd4193b5702478615a2aec&interface=public®ion_id=RegionOne => generated 194 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:54.637735 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664df705-e6ac-4d33-9932-74f558203810 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:54.728776 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:54.735355 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a245d7-8257-40ab-acdf-7c694f6796eb None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:54.754900 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664df705-e6ac-4d33-9932-74f558203810 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:54.774408 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acdecd7-4753-4754-aed6-0604639f6616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:54.775296 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acdecd7-4753-4754-aed6-0604639f6616 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:54.775680 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acdecd7-4753-4754-aed6-0604639f6616 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:54.776067 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acdecd7-4753-4754-aed6-0604639f6616 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:54.778825 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:54.784766 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acdecd7-4753-4754-aed6-0604639f6616 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:54.786566 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acdecd7-4753-4754-aed6-0604639f6616 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:54.796203 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e1aa75-0ebc-4ee5-8d3a-7d2b77512d67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:54.796829 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e1aa75-0ebc-4ee5-8d3a-7d2b77512d67 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:54.797111 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e1aa75-0ebc-4ee5-8d3a-7d2b77512d67 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:54.797414 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e1aa75-0ebc-4ee5-8d3a-7d2b77512d67 None admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:54.798175 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e1aa75-0ebc-4ee5-8d3a-7d2b77512d67 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:54.798474 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e1aa75-0ebc-4ee5-8d3a-7d2b77512d67 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:54.799906 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e1aa75-0ebc-4ee5-8d3a-7d2b77512d67 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:54.803331 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-8acdecd7-4753-4754-aed6-0604639f6616 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:31:54.803331 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:54.803331 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:54.803331 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:54.803331 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:54.803331 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:54.803331 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:54.803331 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:54.803331 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:54.803331 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:54.803331 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:54.803331 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:54.803331 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 17:31:54.803331 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 17:31:54.803331 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 17:31:54.804207 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 17:31:54.804207 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:54.804207 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:54.804207 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:54.804207 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:54.804207 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:54.804207 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:54.804207 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:54.804207 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:54.804207 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:54.804207 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:54.804207 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:54.804207 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:54.804207 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:54.804207 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:54.804207 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:54.804207 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:54.805172 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 17:31:54.805172 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 17:31:54.805172 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 17:31:54.805172 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 17:31:54.805172 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 17:31:54.805172 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 17:31:54.805172 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:31:54.805172 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:31:54.813302 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 375/768] 10.222.0.89 () {56 vars in 1111 bytes} [Thu Jul 28 17:31:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:31:54.814383 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 394/769] 10.222.0.89 () {56 vars in 1148 bytes} [Thu Jul 28 17:31:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:54.815236 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e1aa75-0ebc-4ee5-8d3a-7d2b77512d67 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:54.815357 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acdecd7-4753-4754-aed6-0604639f6616 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:54.938310 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e1aa75-0ebc-4ee5-8d3a-7d2b77512d67 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:54.956480 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acdecd7-4753-4754-aed6-0604639f6616 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:54.985850 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f0b09f-9f5b-43c5-ad5d-78a0775a3bfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:54.986479 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f0b09f-9f5b-43c5-ad5d-78a0775a3bfa None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:54.986756 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f0b09f-9f5b-43c5-ad5d-78a0775a3bfa None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:54.987025 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f0b09f-9f5b-43c5-ad5d-78a0775a3bfa None admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:54.987803 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f0b09f-9f5b-43c5-ad5d-78a0775a3bfa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:54.988107 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f0b09f-9f5b-43c5-ad5d-78a0775a3bfa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:54.989556 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f0b09f-9f5b-43c5-ad5d-78a0775a3bfa None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:55.004042 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d415434-67bd-470f-8376-5a694abf1da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:55.004782 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d415434-67bd-470f-8376-5a694abf1da3 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:55.005205 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d415434-67bd-470f-8376-5a694abf1da3 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:55.005497 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d415434-67bd-470f-8376-5a694abf1da3 None admin] PATH_INFO: `/v3/services` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:55.012648 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 395/770] 10.222.0.89 () {56 vars in 1148 bytes} [Thu Jul 28 17:31:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:55.012820 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d415434-67bd-470f-8376-5a694abf1da3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:55.012912 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f0b09f-9f5b-43c5-ad5d-78a0775a3bfa None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:55.013019 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d415434-67bd-470f-8376-5a694abf1da3 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:55.018094 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 376/771] 10.222.0.89 () {56 vars in 1087 bytes} [Thu Jul 28 17:31:54 2022] GET /identity/v3/services => generated 772 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:55.022835 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ef7984-e51a-4cfd-97e2-ce6961e0f335 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:55.022835 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ef7984-e51a-4cfd-97e2-ce6961e0f335 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:55.022835 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ef7984-e51a-4cfd-97e2-ce6961e0f335 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:55.022835 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 377/772] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:31:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:31:55.022835 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ef7984-e51a-4cfd-97e2-ce6961e0f335 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:55.056692 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:55.085597 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:55.172759 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f0b09f-9f5b-43c5-ad5d-78a0775a3bfa None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:55.178824 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ef7984-e51a-4cfd-97e2-ce6961e0f335 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:55.215773 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9c749f-4dbd-4f87-a13a-d72151158ea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:55.217398 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9c749f-4dbd-4f87-a13a-d72151158ea5 None admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:55.217873 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9c749f-4dbd-4f87-a13a-d72151158ea5 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:55.218180 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9c749f-4dbd-4f87-a13a-d72151158ea5 None admin] PATH_INFO: `/v3/projects/04434e8535a64044996316f1d17dc0bc/users/d10b49b29df94404a71f43273839c252/roles/73e457231a824ea98c28d22610be640c` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:55.222744 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9c749f-4dbd-4f87-a13a-d72151158ea5 None admin] RBAC: Authorizing `identity:create_grant(project_id=04434e8535a64044996316f1d17dc0bc, user_id=d10b49b29df94404a71f43273839c252, role_id=73e457231a824ea98c28d22610be640c)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:55.222744 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9c749f-4dbd-4f87-a13a-d72151158ea5 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:55.227693 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5e5e57-312b-4c9b-b32f-19919d7d2e69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:55.228403 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5e5e57-312b-4c9b-b32f-19919d7d2e69 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:55.228724 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5e5e57-312b-4c9b-b32f-19919d7d2e69 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:55.229102 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5e5e57-312b-4c9b-b32f-19919d7d2e69 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:55.229955 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5e5e57-312b-4c9b-b32f-19919d7d2e69 None admin] RBAC: Adding query filter params (role.id=0a9ea6e6d6f94955a5dc949b0746a3f1, scope.project.id=04434e8535a64044996316f1d17dc0bc, user.id=c44387d13a694ee6bb47a13906003330) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:55.230387 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5e5e57-312b-4c9b-b32f-19919d7d2e69 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:55.231852 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5e5e57-312b-4c9b-b32f-19919d7d2e69 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:55.241030 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 378/773] 10.222.0.89 () {56 vars in 1374 bytes} [Thu Jul 28 17:31:55 2022] GET /identity/v3/role_assignments?user.id=c44387d13a694ee6bb47a13906003330&scope.project.id=04434e8535a64044996316f1d17dc0bc&role.id=0a9ea6e6d6f94955a5dc949b0746a3f1 => generated 259 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:55.242591 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58191fdb-f713-4945-9472-2b5220a41b9d None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:55.242952 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58191fdb-f713-4945-9472-2b5220a41b9d None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:55.243293 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58191fdb-f713-4945-9472-2b5220a41b9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:55.246107 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 396/774] 10.222.0.89 () {58 vars in 1439 bytes} [Thu Jul 28 17:31:55 2022] PUT /identity/v3/projects/04434e8535a64044996316f1d17dc0bc/users/d10b49b29df94404a71f43273839c252/roles/73e457231a824ea98c28d22610be640c => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:31:55.282511 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-58191fdb-f713-4945-9472-2b5220a41b9d None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:31:55.283882 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-58191fdb-f713-4945-9472-2b5220a41b9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:31:55.318976 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7873ca-35f2-4e17-80bf-e6bec4f3119c None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:55.319262 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7873ca-35f2-4e17-80bf-e6bec4f3119c None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:55.319542 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7873ca-35f2-4e17-80bf-e6bec4f3119c None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:55.325308 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 397/775] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:31:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 17:31:55.327718 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dac72db-4f99-45ed-a5eb-0ad67b0166f5 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:55.327995 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dac72db-4f99-45ed-a5eb-0ad67b0166f5 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:55.328401 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dac72db-4f99-45ed-a5eb-0ad67b0166f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:55.349283 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-5dac72db-4f99-45ed-a5eb-0ad67b0166f5 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:31:55.350703 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-5dac72db-4f99-45ed-a5eb-0ad67b0166f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:31:55.405211 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 379/776] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:31:55 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:31:55.428067 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58191fdb-f713-4945-9472-2b5220a41b9d None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:55.446074 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:55.476807 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 398/777] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:31:55 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:31:55.501979 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dac72db-4f99-45ed-a5eb-0ad67b0166f5 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:55.543558 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:55.581308 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58191fdb-f713-4945-9472-2b5220a41b9d None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:55.620383 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dac72db-4f99-45ed-a5eb-0ad67b0166f5 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:55.622235 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385f5d24-ac5e-4b75-a13f-6a371b0f3138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:55.623092 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385f5d24-ac5e-4b75-a13f-6a371b0f3138 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:55.623373 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385f5d24-ac5e-4b75-a13f-6a371b0f3138 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:55.623649 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385f5d24-ac5e-4b75-a13f-6a371b0f3138 None admin] PATH_INFO: `/v3/services/network` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:55.624407 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385f5d24-ac5e-4b75-a13f-6a371b0f3138 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:55.625835 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385f5d24-ac5e-4b75-a13f-6a371b0f3138 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:55.636078 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-385f5d24-ac5e-4b75-a13f-6a371b0f3138 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 17:31:55.636078 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:55.636078 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 17:31:55.636078 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 17:31:55.636078 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 17:31:55.636078 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 17:31:55.636078 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 17:31:55.636078 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:31:55.636078 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 17:31:55.636078 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:31:55.636078 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 17:31:55.636078 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:31:55.636078 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:55.636078 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:55.636078 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:55.636078 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:55.636078 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:55.637121 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:55.637121 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:55.637121 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:55.637121 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:55.637121 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:55.637121 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:55.637121 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 17:31:55.637121 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 17:31:55.637121 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 17:31:55.637121 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 17:31:55.637121 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:55.637121 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:55.637121 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:55.637121 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:55.637121 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:55.637121 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:55.637998 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:55.637998 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:55.637998 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:55.637998 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:55.637998 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:55.637998 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:55.637998 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:55.637998 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:55.637998 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:55.637998 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:55.637998 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 17:31:55.637998 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:31:55.637998 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 17:31:55.637998 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:31:55.637998 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 380/778] 10.222.0.89 () {56 vars in 1111 bytes} [Thu Jul 28 17:31:55 2022] GET /identity/v3/services/network => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:31:55.644029 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385f5d24-ac5e-4b75-a13f-6a371b0f3138 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:55.658792 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12899f8b-c059-45ac-86fb-d40dd110a265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:55.659433 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12899f8b-c059-45ac-86fb-d40dd110a265 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:55.659708 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12899f8b-c059-45ac-86fb-d40dd110a265 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:55.659979 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12899f8b-c059-45ac-86fb-d40dd110a265 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:55.671116 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12899f8b-c059-45ac-86fb-d40dd110a265 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:31:55.676101 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-12899f8b-c059-45ac-86fb-d40dd110a265 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:31:55.676101 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:55.676101 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:55.676101 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:55.676101 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:55.676101 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:55.676101 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:55.676101 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:55.676101 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:55.676101 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:55.676101 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:55.676101 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:55.676101 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:31:55.676101 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:31:55.676101 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:31:55.676101 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:31:55.678236 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:31:55.678236 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:31:55.678236 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:31:55.678236 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:31:55.678236 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:55.678236 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:55.678236 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:55.678236 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:55.678236 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:55.678236 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:55.678236 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:55.678236 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:55.678236 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:55.678236 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:55.678236 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:55.678236 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:55.680370 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:55.680370 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:55.680370 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:55.680370 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:55.680370 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:31:55.680370 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:31:55.680370 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:31:55.680370 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:31:55.680370 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:31:55.680370 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:31:55.680370 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:31:55.680370 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:31:55.680370 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 399/779] 10.222.0.89 () {56 vars in 1096 bytes} [Thu Jul 28 17:31:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:31:55.686755 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12899f8b-c059-45ac-86fb-d40dd110a265 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:55.738115 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:55.765050 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385f5d24-ac5e-4b75-a13f-6a371b0f3138 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:55.780706 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:55.790514 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12899f8b-c059-45ac-86fb-d40dd110a265 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:55.803982 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b328d28b-4603-461a-af16-be5746a45a91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:55.804614 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b328d28b-4603-461a-af16-be5746a45a91 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:55.804899 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b328d28b-4603-461a-af16-be5746a45a91 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:55.805190 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b328d28b-4603-461a-af16-be5746a45a91 None admin] PATH_INFO: `/v3/services` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:55.805985 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b328d28b-4603-461a-af16-be5746a45a91 None admin] RBAC: Adding query filter params (name=network) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:55.806293 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b328d28b-4603-461a-af16-be5746a45a91 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:55.813722 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b328d28b-4603-461a-af16-be5746a45a91 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:55.817188 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 381/780] 10.222.0.89 () {56 vars in 1112 bytes} [Thu Jul 28 17:31:55 2022] GET /identity/v3/services?name=network => generated 124 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:55.819827 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b328d28b-4603-461a-af16-be5746a45a91 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:55.832245 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c66bf9e-4159-4cbd-a939-4b0cb6bd6cfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:55.832968 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c66bf9e-4159-4cbd-a939-4b0cb6bd6cfb None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:55.833371 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c66bf9e-4159-4cbd-a939-4b0cb6bd6cfb None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:55.833755 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c66bf9e-4159-4cbd-a939-4b0cb6bd6cfb None admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:55.834576 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c66bf9e-4159-4cbd-a939-4b0cb6bd6cfb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:55.834971 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c66bf9e-4159-4cbd-a939-4b0cb6bd6cfb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:55.836422 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c66bf9e-4159-4cbd-a939-4b0cb6bd6cfb None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:55.846688 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 400/781] 10.222.0.89 () {56 vars in 1099 bytes} [Thu Jul 28 17:31:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:31:55.852314 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c66bf9e-4159-4cbd-a939-4b0cb6bd6cfb None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:55.934874 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b328d28b-4603-461a-af16-be5746a45a91 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:55.961331 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c66bf9e-4159-4cbd-a939-4b0cb6bd6cfb None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:55.975015 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53c1f62-ce7f-4020-b294-c439bba22049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:55.975805 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53c1f62-ce7f-4020-b294-c439bba22049 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:55.976094 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53c1f62-ce7f-4020-b294-c439bba22049 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:55.976368 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53c1f62-ce7f-4020-b294-c439bba22049 None admin] PATH_INFO: `/v3/services` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:55.977185 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53c1f62-ce7f-4020-b294-c439bba22049 None admin] RBAC: Adding query filter params (type=network) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:55.977516 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53c1f62-ce7f-4020-b294-c439bba22049 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:55.979097 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53c1f62-ce7f-4020-b294-c439bba22049 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:55.985337 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 382/782] 10.222.0.89 () {56 vars in 1112 bytes} [Thu Jul 28 17:31:55 2022] GET /identity/v3/services?type=network => generated 350 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:31:55.991273 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53c1f62-ce7f-4020-b294-c439bba22049 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:56.000021 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146963ec-f375-494e-a5f1-5eb82db46efc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:56.000834 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146963ec-f375-494e-a5f1-5eb82db46efc None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:56.001249 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146963ec-f375-494e-a5f1-5eb82db46efc None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:56.001625 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146963ec-f375-494e-a5f1-5eb82db46efc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:56.002871 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:56.014886 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146963ec-f375-494e-a5f1-5eb82db46efc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:56.016536 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146963ec-f375-494e-a5f1-5eb82db46efc None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:56.031884 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-146963ec-f375-494e-a5f1-5eb82db46efc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:31:56.031884 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:56.031884 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:56.031884 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:56.031884 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:56.031884 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:56.031884 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:56.031884 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:56.031884 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:56.031884 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:56.031884 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:56.031884 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:56.031884 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:31:56.031884 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:31:56.031884 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:31:56.033017 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:31:56.033017 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:56.033017 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:56.033017 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:56.033017 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:56.033017 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:56.033017 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:56.033017 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:56.033017 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:56.033017 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:56.033017 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:56.033017 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:56.033017 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:56.033017 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:56.033017 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:56.033017 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:56.033017 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:56.034189 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:31:56.034189 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:31:56.034189 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:31:56.034189 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:31:56.035579 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 401/783] 10.222.0.89 () {56 vars in 1108 bytes} [Thu Jul 28 17:31:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:31:56.041090 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146963ec-f375-494e-a5f1-5eb82db46efc None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:56.045711 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:56.117311 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53c1f62-ce7f-4020-b294-c439bba22049 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:56.148622 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146963ec-f375-494e-a5f1-5eb82db46efc None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:56.163705 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7880765-51f6-45aa-a072-b44724b7ce51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:56.164578 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7880765-51f6-45aa-a072-b44724b7ce51 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:56.165010 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7880765-51f6-45aa-a072-b44724b7ce51 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:56.165454 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7880765-51f6-45aa-a072-b44724b7ce51 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:56.166564 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7880765-51f6-45aa-a072-b44724b7ce51 None admin] RBAC: Adding query filter params (interface=public, service_id=16b92e90e4294eb28872204f3f10c23d, region_id=RegionOne) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:56.167193 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7880765-51f6-45aa-a072-b44724b7ce51 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:56.168732 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7880765-51f6-45aa-a072-b44724b7ce51 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:56.185378 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f224d2-fcf8-4a24-9783-0d619bbfb0b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:56.187099 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f224d2-fcf8-4a24-9783-0d619bbfb0b7 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:56.187507 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f224d2-fcf8-4a24-9783-0d619bbfb0b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:56.187950 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f224d2-fcf8-4a24-9783-0d619bbfb0b7 None admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:56.188339 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 383/784] 10.222.0.89 () {56 vars in 1251 bytes} [Thu Jul 28 17:31:55 2022] GET /identity/v3/endpoints?service_id=16b92e90e4294eb28872204f3f10c23d&interface=public®ion_id=RegionOne => generated 194 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:31:56.192755 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f224d2-fcf8-4a24-9783-0d619bbfb0b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:56.192755 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f224d2-fcf8-4a24-9783-0d619bbfb0b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:56.192755 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f224d2-fcf8-4a24-9783-0d619bbfb0b7 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:56.193922 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7880765-51f6-45aa-a072-b44724b7ce51 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:56.204505 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 402/785] 10.222.0.89 () {56 vars in 1109 bytes} [Thu Jul 28 17:31:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:31:56.209857 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f224d2-fcf8-4a24-9783-0d619bbfb0b7 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:56.276201 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:56.321680 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f224d2-fcf8-4a24-9783-0d619bbfb0b7 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:56.359570 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bfd8d1-89f0-4efa-8462-694a00967215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:56.360422 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bfd8d1-89f0-4efa-8462-694a00967215 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:56.360822 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bfd8d1-89f0-4efa-8462-694a00967215 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:56.361234 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bfd8d1-89f0-4efa-8462-694a00967215 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:56.369581 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bfd8d1-89f0-4efa-8462-694a00967215 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:56.371469 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bfd8d1-89f0-4efa-8462-694a00967215 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:56.386650 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-a9bfd8d1-89f0-4efa-8462-694a00967215 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 28 17:31:56.386650 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:56.386650 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:56.386650 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:56.386650 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:56.386650 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:56.386650 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:56.386650 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:56.386650 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:56.386650 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:56.386650 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:56.386650 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:56.386650 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 17:31:56.386650 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 17:31:56.386650 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 17:31:56.386650 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 17:31:56.387714 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:56.387714 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:56.387714 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 17:31:56.387714 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:31:56.387714 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 17:31:56.387714 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:31:56.387714 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:56.387714 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:56.387714 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:56.387714 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:56.387714 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:56.387714 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:56.387714 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:56.387714 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:56.387714 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:56.387714 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:56.387714 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:56.387714 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:56.389271 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:56.389271 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:56.389271 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 17:31:56.389271 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 17:31:56.389271 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 17:31:56.389271 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 17:31:56.389271 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 17:31:56.389271 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 17:31:56.389271 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 28 17:31:56.389271 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:31:56.389271 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 403/786] 10.222.0.89 () {56 vars in 1096 bytes} [Thu Jul 28 17:31:56 2022] GET /identity/v3/users/swift => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:31:56.395516 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bfd8d1-89f0-4efa-8462-694a00967215 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:56.433492 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:56.487247 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7880765-51f6-45aa-a072-b44724b7ce51 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:56.551264 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26774c1b-d343-4915-aade-dd6050050527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:56.551812 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26774c1b-d343-4915-aade-dd6050050527 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:56.552754 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26774c1b-d343-4915-aade-dd6050050527 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:56.553028 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26774c1b-d343-4915-aade-dd6050050527 None admin] PATH_INFO: `/v3/services` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:56.553730 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26774c1b-d343-4915-aade-dd6050050527 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:56.555099 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26774c1b-d343-4915-aade-dd6050050527 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:56.562928 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bfd8d1-89f0-4efa-8462-694a00967215 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:56.567924 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 384/787] 10.222.0.89 () {56 vars in 1087 bytes} [Thu Jul 28 17:31:56 2022] GET /identity/v3/services => generated 772 bytes in 374 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:56.605580 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:56.628280 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04fac40-e66e-4f83-8486-563407b33a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:56.628926 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04fac40-e66e-4f83-8486-563407b33a55 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:56.629225 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04fac40-e66e-4f83-8486-563407b33a55 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:56.629502 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04fac40-e66e-4f83-8486-563407b33a55 None admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:56.630375 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04fac40-e66e-4f83-8486-563407b33a55 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:56.630689 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04fac40-e66e-4f83-8486-563407b33a55 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:56.632157 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04fac40-e66e-4f83-8486-563407b33a55 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:56.661288 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 404/788] 10.222.0.89 () {56 vars in 1135 bytes} [Thu Jul 28 17:31:56 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:31:56.667046 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26774c1b-d343-4915-aade-dd6050050527 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:56.795625 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26774c1b-d343-4915-aade-dd6050050527 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:56.834656 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0e2d53-fe43-475f-a947-f7d7fa552c92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:56.835313 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0e2d53-fe43-475f-a947-f7d7fa552c92 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:56.835592 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0e2d53-fe43-475f-a947-f7d7fa552c92 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:56.835934 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0e2d53-fe43-475f-a947-f7d7fa552c92 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:56.837930 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:56.848962 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0e2d53-fe43-475f-a947-f7d7fa552c92 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:56.850455 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0e2d53-fe43-475f-a947-f7d7fa552c92 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:56.866702 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-9d0e2d53-fe43-475f-a947-f7d7fa552c92 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:31:56.866702 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:56.866702 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:56.866702 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:56.866702 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:56.866702 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:56.866702 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:56.866702 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:56.866702 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:56.866702 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:56.866702 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:56.866702 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:56.866702 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:31:56.866702 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:31:56.866702 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:31:56.867773 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:31:56.867773 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:56.867773 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:56.867773 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:56.867773 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:56.867773 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:56.867773 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:56.867773 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:56.867773 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:56.867773 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:56.867773 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:56.867773 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:56.867773 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:56.867773 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:56.867773 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:56.867773 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:56.867773 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:56.868740 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:31:56.868740 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:31:56.868740 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:31:56.868740 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:31:56.868740 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 385/789] 10.222.0.89 () {56 vars in 1108 bytes} [Thu Jul 28 17:31:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:31:56.875821 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04fac40-e66e-4f83-8486-563407b33a55 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:57.001676 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04fac40-e66e-4f83-8486-563407b33a55 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:57.054955 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0731087-5736-4120-b8b8-e0a393a1f822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:57.055590 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0731087-5736-4120-b8b8-e0a393a1f822 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:57.055952 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0731087-5736-4120-b8b8-e0a393a1f822 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:57.056230 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0731087-5736-4120-b8b8-e0a393a1f822 None admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:57.056977 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0731087-5736-4120-b8b8-e0a393a1f822 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:57.057355 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0731087-5736-4120-b8b8-e0a393a1f822 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:57.058732 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0731087-5736-4120-b8b8-e0a393a1f822 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:57.076681 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 405/790] 10.222.0.89 () {56 vars in 1109 bytes} [Thu Jul 28 17:31:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:57.082788 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0e2d53-fe43-475f-a947-f7d7fa552c92 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:57.219848 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0e2d53-fe43-475f-a947-f7d7fa552c92 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:57.255373 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd459d0f-b06b-4fa4-8a2b-778dfd6ee88b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:57.256410 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd459d0f-b06b-4fa4-8a2b-778dfd6ee88b None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:57.256899 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd459d0f-b06b-4fa4-8a2b-778dfd6ee88b None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:57.257422 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd459d0f-b06b-4fa4-8a2b-778dfd6ee88b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:57.264642 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd459d0f-b06b-4fa4-8a2b-778dfd6ee88b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:57.266503 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd459d0f-b06b-4fa4-8a2b-778dfd6ee88b None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:57.279110 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-fd459d0f-b06b-4fa4-8a2b-778dfd6ee88b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:31:57.279110 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:57.279110 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:57.279110 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:57.279110 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:57.279110 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:57.279110 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:57.279110 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:57.279110 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:57.279110 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:57.279110 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:57.279110 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:57.279110 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 17:31:57.279110 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 17:31:57.279110 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 17:31:57.280767 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 17:31:57.280767 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:57.280767 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:57.280767 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:57.280767 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:57.280767 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:57.280767 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:57.280767 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:57.280767 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:57.280767 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:57.280767 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:57.280767 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:57.280767 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:57.280767 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:57.280767 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:57.280767 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:57.280767 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:57.281728 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 17:31:57.281728 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 17:31:57.281728 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 17:31:57.281728 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 17:31:57.281728 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 17:31:57.281728 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 17:31:57.281728 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:31:57.281728 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:31:57.284151 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 386/791] 10.222.0.89 () {56 vars in 1111 bytes} [Thu Jul 28 17:31:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:31:57.290303 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0731087-5736-4120-b8b8-e0a393a1f822 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:57.339550 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:57.402742 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0731087-5736-4120-b8b8-e0a393a1f822 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:57.440305 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff82e46f-ab09-45f9-8ebf-af41dfdce2f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:57.440927 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff82e46f-ab09-45f9-8ebf-af41dfdce2f0 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:57.441234 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff82e46f-ab09-45f9-8ebf-af41dfdce2f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:57.441513 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff82e46f-ab09-45f9-8ebf-af41dfdce2f0 None admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:57.442265 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff82e46f-ab09-45f9-8ebf-af41dfdce2f0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:57.442570 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff82e46f-ab09-45f9-8ebf-af41dfdce2f0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:57.444044 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff82e46f-ab09-45f9-8ebf-af41dfdce2f0 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:57.461485 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 406/792] 10.222.0.89 () {56 vars in 1148 bytes} [Thu Jul 28 17:31:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:31:57.463433 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd459d0f-b06b-4fa4-8a2b-778dfd6ee88b None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:57.526329 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:57.598101 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd459d0f-b06b-4fa4-8a2b-778dfd6ee88b None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:57.644238 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763b75c1-b9c5-4fee-8d11-0ea588b8b128 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:57.644562 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763b75c1-b9c5-4fee-8d11-0ea588b8b128 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:57.644794 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763b75c1-b9c5-4fee-8d11-0ea588b8b128 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:57.646461 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 407/793] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:31:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:31:57.647518 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bb1db1-95c1-4b14-9e4b-ad0cad533450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:57.648204 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bb1db1-95c1-4b14-9e4b-ad0cad533450 None admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:57.648481 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bb1db1-95c1-4b14-9e4b-ad0cad533450 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:57.648754 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bb1db1-95c1-4b14-9e4b-ad0cad533450 None admin] PATH_INFO: `/v3/projects/04434e8535a64044996316f1d17dc0bc/users/3c59473b01bf4433ba3b111114fc048b/roles/73e457231a824ea98c28d22610be640c` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:57.650941 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccd5b2a-f6af-4890-95de-a212c01ab307 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:57.651289 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccd5b2a-f6af-4890-95de-a212c01ab307 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:57.651566 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccd5b2a-f6af-4890-95de-a212c01ab307 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:57.657983 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bb1db1-95c1-4b14-9e4b-ad0cad533450 None admin] RBAC: Authorizing `identity:create_grant(project_id=04434e8535a64044996316f1d17dc0bc, user_id=3c59473b01bf4433ba3b111114fc048b, role_id=73e457231a824ea98c28d22610be640c)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:57.659448 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bb1db1-95c1-4b14-9e4b-ad0cad533450 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:57.677776 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-9ccd5b2a-f6af-4890-95de-a212c01ab307 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:31:57.679064 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-9ccd5b2a-f6af-4890-95de-a212c01ab307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:31:57.688567 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 387/794] 10.222.0.89 () {58 vars in 1439 bytes} [Thu Jul 28 17:31:57 2022] PUT /identity/v3/projects/04434e8535a64044996316f1d17dc0bc/users/3c59473b01bf4433ba3b111114fc048b/roles/73e457231a824ea98c28d22610be640c => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:31:57.818141 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 408/795] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:31:57 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:31:57.851436 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bb1db1-95c1-4b14-9e4b-ad0cad533450 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:57.903982 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11172342-5410-4b39-9bd0-5595e8d50784 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:57.905058 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11172342-5410-4b39-9bd0-5595e8d50784 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:57.905282 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11172342-5410-4b39-9bd0-5595e8d50784 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:57.907137 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 409/796] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:31:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:31:57.918705 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003345d2-31bc-4a45-b0f5-adf3a174e901 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:57.919138 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003345d2-31bc-4a45-b0f5-adf3a174e901 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:57.919414 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003345d2-31bc-4a45-b0f5-adf3a174e901 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:57.956094 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-003345d2-31bc-4a45-b0f5-adf3a174e901 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:31:57.958098 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-003345d2-31bc-4a45-b0f5-adf3a174e901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:31:57.979789 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bb1db1-95c1-4b14-9e4b-ad0cad533450 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:57.990686 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:58.000098 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:58.030972 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4683921-eefb-4f62-a538-340ad6ddf746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:58.032006 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4683921-eefb-4f62-a538-340ad6ddf746 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:58.032290 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4683921-eefb-4f62-a538-340ad6ddf746 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:58.032565 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4683921-eefb-4f62-a538-340ad6ddf746 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:58.033256 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4683921-eefb-4f62-a538-340ad6ddf746 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:58.034799 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4683921-eefb-4f62-a538-340ad6ddf746 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:58.044643 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-d4683921-eefb-4f62-a538-340ad6ddf746 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 17:31:58.044643 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:58.044643 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 17:31:58.044643 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 17:31:58.044643 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 17:31:58.044643 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 17:31:58.044643 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 17:31:58.044643 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:31:58.044643 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 17:31:58.044643 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:31:58.044643 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 17:31:58.044643 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:31:58.044643 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:58.044643 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:58.044643 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:58.044643 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:58.046588 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:58.046588 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:58.046588 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:58.046588 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:58.046588 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:58.046588 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:58.046588 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:58.046588 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 17:31:58.046588 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 17:31:58.046588 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 17:31:58.046588 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 17:31:58.046588 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:58.046588 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:58.046588 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:58.046588 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:58.046588 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:58.047536 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:58.047536 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:58.047536 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:58.047536 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:58.047536 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:58.047536 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:58.047536 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:58.047536 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:58.047536 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:58.047536 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:58.047536 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:58.047536 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 17:31:58.047536 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:31:58.047536 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 17:31:58.047536 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:31:58.047536 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 388/797] 10.222.0.89 () {56 vars in 1129 bytes} [Thu Jul 28 17:31:57 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:31:58.054793 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4683921-eefb-4f62-a538-340ad6ddf746 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:58.062300 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 410/798] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:31:57 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 17:31:58.175553 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd810bb-c323-484f-8dcc-bbdb459ab869 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:58.176314 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd810bb-c323-484f-8dcc-bbdb459ab869 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:58.176910 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd810bb-c323-484f-8dcc-bbdb459ab869 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:58.179403 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 411/799] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:31:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:31:58.187641 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd810bb-c323-484f-8dcc-bbdb459ab869 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:58.219273 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4683921-eefb-4f62-a538-340ad6ddf746 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:58.260883 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8988b0c8-638e-4a18-ac07-671df4180c1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:58.261887 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8988b0c8-638e-4a18-ac07-671df4180c1a None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:58.262417 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8988b0c8-638e-4a18-ac07-671df4180c1a None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:58.263018 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8988b0c8-638e-4a18-ac07-671df4180c1a None admin] PATH_INFO: `/v3/services` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:58.264520 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8988b0c8-638e-4a18-ac07-671df4180c1a None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:58.265039 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8988b0c8-638e-4a18-ac07-671df4180c1a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:58.266854 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8988b0c8-638e-4a18-ac07-671df4180c1a None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:58.272342 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 389/800] 10.222.0.89 () {56 vars in 1124 bytes} [Thu Jul 28 17:31:58 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:58.274002 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fbb82d-9095-41a5-807a-9ba09514c136 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:58.274488 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fbb82d-9095-41a5-807a-9ba09514c136 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:58.274971 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fbb82d-9095-41a5-807a-9ba09514c136 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:58.295221 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-34fbb82d-9095-41a5-807a-9ba09514c136 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:31:58.296973 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-34fbb82d-9095-41a5-807a-9ba09514c136 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:31:58.323179 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd810bb-c323-484f-8dcc-bbdb459ab869 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:58.333009 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:58.370487 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0e064b-e1d7-477e-93e7-029d9b3b46de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:58.371125 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0e064b-e1d7-477e-93e7-029d9b3b46de None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:58.371404 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0e064b-e1d7-477e-93e7-029d9b3b46de None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:58.371680 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0e064b-e1d7-477e-93e7-029d9b3b46de None admin] PATH_INFO: `/v3/roles/service` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:58.380897 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 390/801] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:31:58 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:31:58.382400 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fbb82d-9095-41a5-807a-9ba09514c136 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:58.385173 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0e064b-e1d7-477e-93e7-029d9b3b46de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:31:58.391831 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-ee0e064b-e1d7-477e-93e7-029d9b3b46de None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 17:31:58.391831 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:58.391831 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:58.391831 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:58.391831 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:58.391831 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:58.391831 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:58.391831 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:58.391831 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:58.391831 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:58.391831 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:58.391831 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:58.391831 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:31:58.391831 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:31:58.391831 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:31:58.391831 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:31:58.392934 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:31:58.392934 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:31:58.392934 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:31:58.392934 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:31:58.392934 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:58.392934 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:58.392934 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:58.392934 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:58.392934 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:58.392934 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:58.392934 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:58.392934 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:58.392934 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:58.392934 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:58.392934 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:58.392934 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:58.393818 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:58.393818 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:58.393818 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:58.393818 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:58.393818 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:31:58.393818 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:31:58.393818 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:31:58.393818 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:31:58.393818 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:31:58.393818 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:31:58.393818 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 17:31:58.393818 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:31:58.397237 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 412/802] 10.222.0.89 () {56 vars in 1102 bytes} [Thu Jul 28 17:31:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:31:58.399779 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0e064b-e1d7-477e-93e7-029d9b3b46de None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:58.434856 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:58.521742 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fbb82d-9095-41a5-807a-9ba09514c136 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:58.535514 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0e064b-e1d7-477e-93e7-029d9b3b46de None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:58.570238 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a0cdbb-a719-4e94-a0ed-2750e4cd77d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:58.570862 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a0cdbb-a719-4e94-a0ed-2750e4cd77d8 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:58.571138 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a0cdbb-a719-4e94-a0ed-2750e4cd77d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:58.571408 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a0cdbb-a719-4e94-a0ed-2750e4cd77d8 None admin] PATH_INFO: `/v3/services` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:58.572086 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a0cdbb-a719-4e94-a0ed-2750e4cd77d8 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:58.572387 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a0cdbb-a719-4e94-a0ed-2750e4cd77d8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:58.573772 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a0cdbb-a719-4e94-a0ed-2750e4cd77d8 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:58.578142 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4f8e78-0227-464d-b6ec-ae82cd1aa7de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:58.579058 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4f8e78-0227-464d-b6ec-ae82cd1aa7de None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:58.579539 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4f8e78-0227-464d-b6ec-ae82cd1aa7de None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:58.580100 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 391/803] 10.222.0.89 () {56 vars in 1124 bytes} [Thu Jul 28 17:31:58 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:58.580211 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4f8e78-0227-464d-b6ec-ae82cd1aa7de None admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:58.581018 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a0cdbb-a719-4e94-a0ed-2750e4cd77d8 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:58.581571 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4f8e78-0227-464d-b6ec-ae82cd1aa7de None admin] RBAC: Adding query filter params (name=service) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:58.582096 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4f8e78-0227-464d-b6ec-ae82cd1aa7de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:58.584081 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4f8e78-0227-464d-b6ec-ae82cd1aa7de None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:58.605296 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 413/804] 10.222.0.89 () {56 vars in 1103 bytes} [Thu Jul 28 17:31:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:31:58.605296 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4f8e78-0227-464d-b6ec-ae82cd1aa7de None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:58.619270 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:58.703183 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:58.721169 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4f8e78-0227-464d-b6ec-ae82cd1aa7de None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:58.722265 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a0cdbb-a719-4e94-a0ed-2750e4cd77d8 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:58.756880 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20616da-cca4-4369-841c-4c3484288300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:58.757646 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20616da-cca4-4369-841c-4c3484288300 None admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:58.757991 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20616da-cca4-4369-841c-4c3484288300 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:58.758332 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20616da-cca4-4369-841c-4c3484288300 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:58.759114 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20616da-cca4-4369-841c-4c3484288300 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:58.760530 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20616da-cca4-4369-841c-4c3484288300 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:58.768815 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc9403e-f440-4bfe-afa6-f75bd5d6abe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:58.769718 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc9403e-f440-4bfe-afa6-f75bd5d6abe8 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:58.770335 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc9403e-f440-4bfe-afa6-f75bd5d6abe8 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:58.770809 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc9403e-f440-4bfe-afa6-f75bd5d6abe8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:58.784210 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc9403e-f440-4bfe-afa6-f75bd5d6abe8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:31:58.787891 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 414/805] 10.222.0.89 () {60 vars in 1144 bytes} [Thu Jul 28 17:31:58 2022] POST /identity/v3/endpoints => generated 346 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:31:58.789295 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20616da-cca4-4369-841c-4c3484288300 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:58.790450 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-bbc9403e-f440-4bfe-afa6-f75bd5d6abe8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:31:58.790450 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:58.790450 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:58.790450 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:58.790450 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:58.790450 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:58.790450 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:58.790450 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:58.790450 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:58.790450 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:58.790450 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:58.790450 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:58.790450 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:31:58.790450 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:31:58.790450 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:31:58.790450 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:31:58.791414 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:31:58.791414 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:31:58.791414 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:31:58.791414 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:31:58.791414 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:58.791414 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:58.791414 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:58.791414 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:58.791414 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:58.791414 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:58.791414 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:58.791414 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:58.791414 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:58.791414 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:58.791414 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:58.791414 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:58.792291 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:58.792291 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:58.792291 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:58.792291 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:58.792291 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:31:58.792291 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:31:58.792291 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:31:58.792291 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:31:58.792291 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:31:58.792291 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:31:58.792291 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:31:58.792291 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:31:58.796023 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 392/806] 10.222.0.89 () {56 vars in 1096 bytes} [Thu Jul 28 17:31:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:31:58.802199 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc9403e-f440-4bfe-afa6-f75bd5d6abe8 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:58.987344 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20616da-cca4-4369-841c-4c3484288300 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:59.008086 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc9403e-f440-4bfe-afa6-f75bd5d6abe8 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:59.025533 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8209779f-9710-4ab0-b90e-d541069cf731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:59.026462 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8209779f-9710-4ab0-b90e-d541069cf731 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:59.026841 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8209779f-9710-4ab0-b90e-d541069cf731 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:59.027235 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8209779f-9710-4ab0-b90e-d541069cf731 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:59.029572 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:59.030973 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:59.043560 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8209779f-9710-4ab0-b90e-d541069cf731 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:59.046374 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8209779f-9710-4ab0-b90e-d541069cf731 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:59.055668 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2c24d7-c450-4d60-ac95-aaf3be0aa39a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:59.056606 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2c24d7-c450-4d60-ac95-aaf3be0aa39a None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:59.057058 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2c24d7-c450-4d60-ac95-aaf3be0aa39a None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:59.057615 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2c24d7-c450-4d60-ac95-aaf3be0aa39a None admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:59.058487 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2c24d7-c450-4d60-ac95-aaf3be0aa39a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:59.058953 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2c24d7-c450-4d60-ac95-aaf3be0aa39a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:59.061117 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2c24d7-c450-4d60-ac95-aaf3be0aa39a None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:59.064770 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-8209779f-9710-4ab0-b90e-d541069cf731 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:31:59.064770 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:59.064770 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:59.064770 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:59.064770 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:59.064770 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:59.064770 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:59.064770 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:59.064770 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:59.064770 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:59.064770 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:59.064770 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:59.064770 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:31:59.064770 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:31:59.064770 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:31:59.065881 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:31:59.065881 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:59.065881 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:59.065881 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:59.065881 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:59.065881 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:59.065881 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:59.065881 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:59.065881 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:59.065881 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:59.065881 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:59.065881 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:59.065881 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:59.065881 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:59.065881 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:59.065881 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:59.065881 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:59.067002 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:31:59.067002 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:31:59.067002 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:31:59.067002 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:31:59.068513 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 415/807] 10.222.0.89 () {56 vars in 1108 bytes} [Thu Jul 28 17:31:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:31:59.073651 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 393/808] 10.222.0.89 () {56 vars in 1099 bytes} [Thu Jul 28 17:31:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:59.075016 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8209779f-9710-4ab0-b90e-d541069cf731 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:59.086198 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2c24d7-c450-4d60-ac95-aaf3be0aa39a None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:59.204121 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8209779f-9710-4ab0-b90e-d541069cf731 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:59.231050 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2c24d7-c450-4d60-ac95-aaf3be0aa39a None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:59.241104 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edab4dfc-84bd-4ff9-a654-69ed3a9b3482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:59.241901 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edab4dfc-84bd-4ff9-a654-69ed3a9b3482 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:59.242015 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edab4dfc-84bd-4ff9-a654-69ed3a9b3482 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:59.242218 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edab4dfc-84bd-4ff9-a654-69ed3a9b3482 None admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:59.242911 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edab4dfc-84bd-4ff9-a654-69ed3a9b3482 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:59.243216 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edab4dfc-84bd-4ff9-a654-69ed3a9b3482 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:59.244847 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edab4dfc-84bd-4ff9-a654-69ed3a9b3482 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:59.258363 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 416/809] 10.222.0.89 () {56 vars in 1109 bytes} [Thu Jul 28 17:31:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:59.265515 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edab4dfc-84bd-4ff9-a654-69ed3a9b3482 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:59.282660 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fb67ec-8cce-4e08-acbf-e1900e3e55bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:59.283312 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fb67ec-8cce-4e08-acbf-e1900e3e55bd None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:59.283593 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fb67ec-8cce-4e08-acbf-e1900e3e55bd None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:59.283869 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fb67ec-8cce-4e08-acbf-e1900e3e55bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:59.295576 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fb67ec-8cce-4e08-acbf-e1900e3e55bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:59.297021 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fb67ec-8cce-4e08-acbf-e1900e3e55bd None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:59.312557 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-38fb67ec-8cce-4e08-acbf-e1900e3e55bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:31:59.312557 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:59.312557 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:59.312557 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:59.312557 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:59.312557 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:59.312557 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:59.312557 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:59.312557 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:59.312557 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:59.312557 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:59.312557 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:59.312557 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:31:59.312557 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:31:59.312557 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:31:59.313569 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:31:59.313569 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:59.313569 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:59.313569 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:59.313569 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:59.313569 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:59.313569 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:59.313569 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:59.313569 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:59.313569 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:59.313569 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:59.313569 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:59.313569 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:59.313569 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:59.313569 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:59.313569 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:59.313569 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:59.314498 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:31:59.314498 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:31:59.314498 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:31:59.314498 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:31:59.314498 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 394/810] 10.222.0.89 () {56 vars in 1108 bytes} [Thu Jul 28 17:31:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:31:59.315416 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b9fd80-bd85-4685-8f94-05b8ff620553 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:59.315700 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b9fd80-bd85-4685-8f94-05b8ff620553 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:59.315971 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b9fd80-bd85-4685-8f94-05b8ff620553 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:59.321248 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 395/811] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:31:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:31:59.324080 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b9fd80-bd85-4685-8f94-05b8ff620553 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:59.327661 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:59.337693 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:59.397997 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edab4dfc-84bd-4ff9-a654-69ed3a9b3482 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:59.438105 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa40fe9f-4260-4e8a-9b19-8d025da425be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:59.438752 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa40fe9f-4260-4e8a-9b19-8d025da425be None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:59.439035 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa40fe9f-4260-4e8a-9b19-8d025da425be None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:59.439311 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa40fe9f-4260-4e8a-9b19-8d025da425be None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:59.450411 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa40fe9f-4260-4e8a-9b19-8d025da425be None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:59.453582 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa40fe9f-4260-4e8a-9b19-8d025da425be None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:59.457272 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b9fd80-bd85-4685-8f94-05b8ff620553 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:59.469986 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-aa40fe9f-4260-4e8a-9b19-8d025da425be None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 17:31:59.469986 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:59.469986 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:59.469986 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:59.469986 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:59.469986 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:59.469986 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:59.469986 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:59.469986 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:59.469986 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:59.469986 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:59.469986 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:59.469986 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 17:31:59.469986 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 17:31:59.469986 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 17:31:59.469986 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 17:31:59.470949 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:59.470949 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:59.470949 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 17:31:59.470949 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:31:59.470949 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 17:31:59.470949 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:31:59.470949 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:59.470949 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:59.470949 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:59.470949 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:59.470949 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:59.470949 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:59.470949 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:59.470949 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:59.470949 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:59.470949 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:59.470949 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:59.470949 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:59.471921 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:59.471921 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:59.471921 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 17:31:59.471921 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 17:31:59.471921 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 17:31:59.471921 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 17:31:59.471921 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 17:31:59.471921 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 17:31:59.471921 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 17:31:59.471921 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:31:59.474218 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 417/812] 10.222.0.89 () {56 vars in 1117 bytes} [Thu Jul 28 17:31:59 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:31:59.475562 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a825d3-5acf-463a-bb55-537681ee404b None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:59.476053 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a825d3-5acf-463a-bb55-537681ee404b None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:59.476474 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a825d3-5acf-463a-bb55-537681ee404b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:59.495756 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-e0a825d3-5acf-463a-bb55-537681ee404b None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:31:59.497390 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-e0a825d3-5acf-463a-bb55-537681ee404b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:31:59.502072 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbd159a-d809-425c-8c8e-1a8d540e281b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:59.502777 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbd159a-d809-425c-8c8e-1a8d540e281b None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:59.503164 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbd159a-d809-425c-8c8e-1a8d540e281b None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:59.503539 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbd159a-d809-425c-8c8e-1a8d540e281b None admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:59.504369 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbd159a-d809-425c-8c8e-1a8d540e281b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:59.504777 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbd159a-d809-425c-8c8e-1a8d540e281b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:59.506307 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbd159a-d809-425c-8c8e-1a8d540e281b None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:59.520618 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 396/813] 10.222.0.89 () {56 vars in 1109 bytes} [Thu Jul 28 17:31:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:59.521621 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbd159a-d809-425c-8c8e-1a8d540e281b None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:59.585509 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 418/814] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:31:59 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:31:59.586417 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a825d3-5acf-463a-bb55-537681ee404b None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:59.598905 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:59.648099 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:59.650702 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbd159a-d809-425c-8c8e-1a8d540e281b None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:59.696848 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8871ac0-bcf1-46db-94a5-001557219e94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:59.698033 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8871ac0-bcf1-46db-94a5-001557219e94 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:59.698491 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8871ac0-bcf1-46db-94a5-001557219e94 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:59.698913 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8871ac0-bcf1-46db-94a5-001557219e94 None admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:59.700128 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8871ac0-bcf1-46db-94a5-001557219e94 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:31:59.701151 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8871ac0-bcf1-46db-94a5-001557219e94 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:59.703443 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8871ac0-bcf1-46db-94a5-001557219e94 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:59.721617 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a825d3-5acf-463a-bb55-537681ee404b None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:59.733240 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 397/815] 10.222.0.89 () {56 vars in 1149 bytes} [Thu Jul 28 17:31:59 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:31:59.734459 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8871ac0-bcf1-46db-94a5-001557219e94 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:59.766442 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85bd597-779e-49a8-9eec-71de7dcac0ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:59.767101 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85bd597-779e-49a8-9eec-71de7dcac0ed None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:59.767472 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85bd597-779e-49a8-9eec-71de7dcac0ed None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:59.767810 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85bd597-779e-49a8-9eec-71de7dcac0ed None admin] PATH_INFO: `/v3/users/nova` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:59.775867 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85bd597-779e-49a8-9eec-71de7dcac0ed None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:59.777438 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85bd597-779e-49a8-9eec-71de7dcac0ed None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:59.790774 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-b85bd597-779e-49a8-9eec-71de7dcac0ed None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 17:31:59.790774 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:59.790774 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:59.790774 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:59.790774 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:59.790774 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:59.790774 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:59.790774 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:59.790774 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:59.790774 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:59.790774 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:59.790774 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:59.790774 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 17:31:59.790774 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 17:31:59.790774 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 17:31:59.790774 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 17:31:59.792013 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:59.792013 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:59.792013 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 17:31:59.792013 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:31:59.792013 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 17:31:59.792013 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:31:59.792013 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:59.792013 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:59.792013 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:59.792013 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:59.792013 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:59.792013 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:59.792013 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:59.792013 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:59.792013 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:59.792013 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:59.792013 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:59.792013 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:59.793200 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:59.793200 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:59.793200 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 17:31:59.793200 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 17:31:59.793200 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 17:31:59.793200 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 17:31:59.793200 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 17:31:59.793200 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 17:31:59.793200 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 17:31:59.793200 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:31:59.797363 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 419/816] 10.222.0.89 () {56 vars in 1093 bytes} [Thu Jul 28 17:31:59 2022] GET /identity/v3/users/nova => generated 82 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:31:59.798312 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85bd597-779e-49a8-9eec-71de7dcac0ed None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:59.877200 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8871ac0-bcf1-46db-94a5-001557219e94 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:59.907447 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:59.915132 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:31:59.919807 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85bd597-779e-49a8-9eec-71de7dcac0ed None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:31:59.929096 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7453d78-62d9-4fec-a7fb-d7b9d6bd2f93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:59.929794 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7453d78-62d9-4fec-a7fb-d7b9d6bd2f93 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:59.930074 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7453d78-62d9-4fec-a7fb-d7b9d6bd2f93 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:59.930344 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7453d78-62d9-4fec-a7fb-d7b9d6bd2f93 None admin] PATH_INFO: `/v3/services/network` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:59.931437 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7453d78-62d9-4fec-a7fb-d7b9d6bd2f93 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:59.933251 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7453d78-62d9-4fec-a7fb-d7b9d6bd2f93 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:31:59.945324 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-e7453d78-62d9-4fec-a7fb-d7b9d6bd2f93 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 17:31:59.945324 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:59.945324 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 17:31:59.945324 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 17:31:59.945324 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 17:31:59.945324 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 17:31:59.945324 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 17:31:59.945324 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:31:59.945324 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 17:31:59.945324 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:31:59.945324 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 17:31:59.945324 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:31:59.945324 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:31:59.945324 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:31:59.945324 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:31:59.945324 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:31:59.945324 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:31:59.946345 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:31:59.946345 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:31:59.946345 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:31:59.946345 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:31:59.946345 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:31:59.946345 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:31:59.946345 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 17:31:59.946345 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 17:31:59.946345 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 17:31:59.946345 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 17:31:59.946345 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:31:59.946345 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:31:59.946345 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:31:59.946345 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:31:59.946345 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:31:59.946345 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:31:59.947195 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:31:59.947195 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:31:59.947195 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:31:59.947195 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:31:59.947195 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:31:59.947195 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:31:59.947195 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:31:59.947195 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:31:59.947195 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:31:59.947195 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:31:59.947195 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 17:31:59.947195 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:31:59.947195 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 17:31:59.947195 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:31:59.947195 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 398/817] 10.222.0.89 () {56 vars in 1111 bytes} [Thu Jul 28 17:31:59 2022] GET /identity/v3/services/network => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:31:59.947981 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7453d78-62d9-4fec-a7fb-d7b9d6bd2f93 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:31:59.969487 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158bb262-04c8-4384-9439-d8aa9f8ed6a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:31:59.970134 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158bb262-04c8-4384-9439-d8aa9f8ed6a1 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:31:59.970766 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158bb262-04c8-4384-9439-d8aa9f8ed6a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:31:59.971068 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158bb262-04c8-4384-9439-d8aa9f8ed6a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:31:59.986255 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158bb262-04c8-4384-9439-d8aa9f8ed6a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:31:59.987973 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158bb262-04c8-4384-9439-d8aa9f8ed6a1 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:00.008158 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-158bb262-04c8-4384-9439-d8aa9f8ed6a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:32:00.008158 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:32:00.008158 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:32:00.008158 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:32:00.008158 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:32:00.008158 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:32:00.008158 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:32:00.008158 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:32:00.008158 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:32:00.008158 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:32:00.008158 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:32:00.008158 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:32:00.008158 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:32:00.008158 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:32:00.008158 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:32:00.009107 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:32:00.009107 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:32:00.009107 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:32:00.009107 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:32:00.009107 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:32:00.009107 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:32:00.009107 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:32:00.009107 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:32:00.009107 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( Jul 28 17:32:00.009107 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:32:00.009107 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:32:00.009107 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:32:00.009107 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:32:00.009107 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:32:00.009107 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:32:00.009107 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:32:00.009107 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:32:00.010064 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:32:00.010064 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:32:00.010064 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:32:00.010064 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:32:00.010064 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 420/818] 10.222.0.89 () {56 vars in 1108 bytes} [Thu Jul 28 17:31:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:32:00.010341 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158bb262-04c8-4384-9439-d8aa9f8ed6a1 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:00.066413 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7453d78-62d9-4fec-a7fb-d7b9d6bd2f93 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:00.103530 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54217189-d6ca-4693-bbfb-8b5490fe8fb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:00.104148 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54217189-d6ca-4693-bbfb-8b5490fe8fb2 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:00.104422 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54217189-d6ca-4693-bbfb-8b5490fe8fb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:00.104694 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54217189-d6ca-4693-bbfb-8b5490fe8fb2 None admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:00.105517 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54217189-d6ca-4693-bbfb-8b5490fe8fb2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:32:00.105831 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54217189-d6ca-4693-bbfb-8b5490fe8fb2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:00.107354 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54217189-d6ca-4693-bbfb-8b5490fe8fb2 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:00.124755 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158bb262-04c8-4384-9439-d8aa9f8ed6a1 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:00.125674 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 399/819] 10.222.0.89 () {56 vars in 1133 bytes} [Thu Jul 28 17:31:59 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:32:00.126487 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54217189-d6ca-4693-bbfb-8b5490fe8fb2 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:00.164544 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88977274-839c-4074-a274-df0e1c7f69b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:00.165484 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88977274-839c-4074-a274-df0e1c7f69b7 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:00.165944 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88977274-839c-4074-a274-df0e1c7f69b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:00.166417 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88977274-839c-4074-a274-df0e1c7f69b7 None admin] PATH_INFO: `/v3/services` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:00.167268 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88977274-839c-4074-a274-df0e1c7f69b7 None admin] RBAC: Adding query filter params (name=network) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:32:00.167820 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88977274-839c-4074-a274-df0e1c7f69b7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:00.169301 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88977274-839c-4074-a274-df0e1c7f69b7 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:00.172402 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:00.179820 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 421/820] 10.222.0.89 () {56 vars in 1112 bytes} [Thu Jul 28 17:32:00 2022] GET /identity/v3/services?name=network => generated 124 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:32:00.179820 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92e4e35-9ea5-436f-9680-a33301e23ea8 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:00.179820 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92e4e35-9ea5-436f-9680-a33301e23ea8 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:00.179820 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92e4e35-9ea5-436f-9680-a33301e23ea8 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:00.180197 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 422/821] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:32:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:32:00.184169 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92e4e35-9ea5-436f-9680-a33301e23ea8 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:00.198637 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:00.236666 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54217189-d6ca-4693-bbfb-8b5490fe8fb2 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:00.271605 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c936e59-fe3a-4ee8-9f5d-2dc4b2a3829b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:00.272258 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c936e59-fe3a-4ee8-9f5d-2dc4b2a3829b None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:00.272538 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c936e59-fe3a-4ee8-9f5d-2dc4b2a3829b None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:00.272805 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c936e59-fe3a-4ee8-9f5d-2dc4b2a3829b None admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:00.273525 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c936e59-fe3a-4ee8-9f5d-2dc4b2a3829b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:32:00.273825 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c936e59-fe3a-4ee8-9f5d-2dc4b2a3829b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:00.275185 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c936e59-fe3a-4ee8-9f5d-2dc4b2a3829b None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:00.287919 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 400/822] 10.222.0.89 () {56 vars in 1109 bytes} [Thu Jul 28 17:32:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:32:00.290071 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c936e59-fe3a-4ee8-9f5d-2dc4b2a3829b None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:00.294445 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92e4e35-9ea5-436f-9680-a33301e23ea8 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:00.334510 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76e4b64-2ab7-40fe-84c9-d8efe3c7d4ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:00.335051 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76e4b64-2ab7-40fe-84c9-d8efe3c7d4ce None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:00.335334 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76e4b64-2ab7-40fe-84c9-d8efe3c7d4ce None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:00.335608 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76e4b64-2ab7-40fe-84c9-d8efe3c7d4ce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:00.400050 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c936e59-fe3a-4ee8-9f5d-2dc4b2a3829b None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:00.401761 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:00.407147 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76e4b64-2ab7-40fe-84c9-d8efe3c7d4ce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:00.408587 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76e4b64-2ab7-40fe-84c9-d8efe3c7d4ce None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:00.424848 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:00.441375 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6ac5cf-d1da-4b00-82b5-b94feedc912e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:00.442317 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6ac5cf-d1da-4b00-82b5-b94feedc912e None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:00.442879 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6ac5cf-d1da-4b00-82b5-b94feedc912e None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:00.443161 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6ac5cf-d1da-4b00-82b5-b94feedc912e None admin] PATH_INFO: `/v3/services` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:00.443846 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6ac5cf-d1da-4b00-82b5-b94feedc912e None admin] RBAC: Adding query filter params (type=network) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:32:00.444154 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6ac5cf-d1da-4b00-82b5-b94feedc912e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:00.446500 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6ac5cf-d1da-4b00-82b5-b94feedc912e None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:00.452767 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 401/823] 10.222.0.89 () {56 vars in 1112 bytes} [Thu Jul 28 17:32:00 2022] GET /identity/v3/services?type=network => generated 350 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:32:00.454384 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2f5b20-e94a-4cff-ab6f-aadc66de7359 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:00.454674 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2f5b20-e94a-4cff-ab6f-aadc66de7359 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:00.454953 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2f5b20-e94a-4cff-ab6f-aadc66de7359 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:00.473250 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-9a2f5b20-e94a-4cff-ab6f-aadc66de7359 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:32:00.474453 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-9a2f5b20-e94a-4cff-ab6f-aadc66de7359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:32:00.522782 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 423/824] 10.222.0.89 () {58 vars in 1282 bytes} [Thu Jul 28 17:32:00 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 17:32:00.522782 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76e4b64-2ab7-40fe-84c9-d8efe3c7d4ce None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:00.550372 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 402/825] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:32:00 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:32:00.551029 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2f5b20-e94a-4cff-ab6f-aadc66de7359 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:00.640951 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76e4b64-2ab7-40fe-84c9-d8efe3c7d4ce None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:00.654846 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:00.657528 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:00.662610 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2f5b20-e94a-4cff-ab6f-aadc66de7359 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:00.683444 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfc8d9d-aee0-481c-ba6a-65399b6981de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:00.684090 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfc8d9d-aee0-481c-ba6a-65399b6981de None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:00.684372 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfc8d9d-aee0-481c-ba6a-65399b6981de None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:00.684650 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfc8d9d-aee0-481c-ba6a-65399b6981de None admin] PATH_INFO: `/v3/projects/service` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:00.692228 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfc8d9d-aee0-481c-ba6a-65399b6981de None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:00.693727 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfc8d9d-aee0-481c-ba6a-65399b6981de None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:00.699796 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ebee99-72b5-4542-8126-97875ed86711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:00.700421 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ebee99-72b5-4542-8126-97875ed86711 None admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:00.700691 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ebee99-72b5-4542-8126-97875ed86711 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:00.700964 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ebee99-72b5-4542-8126-97875ed86711 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:00.701810 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ebee99-72b5-4542-8126-97875ed86711 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:00.703149 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ebee99-72b5-4542-8126-97875ed86711 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:00.706615 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-fbfc8d9d-aee0-481c-ba6a-65399b6981de None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:32:00.706615 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:32:00.706615 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:32:00.706615 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:32:00.706615 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:32:00.706615 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:32:00.706615 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:32:00.706615 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:32:00.706615 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:32:00.706615 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:32:00.706615 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:32:00.706615 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:32:00.706615 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 17:32:00.706615 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 17:32:00.706615 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 17:32:00.707499 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 17:32:00.707499 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:32:00.707499 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:32:00.707499 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:32:00.707499 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:32:00.707499 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:32:00.707499 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:32:00.707499 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:32:00.707499 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( Jul 28 17:32:00.707499 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:32:00.707499 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:32:00.707499 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:32:00.707499 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:32:00.707499 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:32:00.707499 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:32:00.707499 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:32:00.707499 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:32:00.708422 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 17:32:00.708422 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 17:32:00.708422 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 17:32:00.708422 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 17:32:00.708422 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 17:32:00.708422 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 17:32:00.708422 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:32:00.708422 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:32:00.708422 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 424/826] 10.222.0.89 () {56 vars in 1111 bytes} [Thu Jul 28 17:32:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:32:00.708953 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfc8d9d-aee0-481c-ba6a-65399b6981de None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:00.730493 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 403/827] 10.222.0.89 () {60 vars in 1144 bytes} [Thu Jul 28 17:32:00 2022] POST /identity/v3/endpoints => generated 327 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:32:00.731645 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ebee99-72b5-4542-8126-97875ed86711 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:00.848216 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfc8d9d-aee0-481c-ba6a-65399b6981de None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:00.875319 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ebee99-72b5-4542-8126-97875ed86711 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:00.889773 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40f4faf-abc9-432b-95a6-f0bdc4e560ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:00.890802 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40f4faf-abc9-432b-95a6-f0bdc4e560ea None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:00.891339 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40f4faf-abc9-432b-95a6-f0bdc4e560ea None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:00.891804 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40f4faf-abc9-432b-95a6-f0bdc4e560ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:00.905274 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40f4faf-abc9-432b-95a6-f0bdc4e560ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:00.907042 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40f4faf-abc9-432b-95a6-f0bdc4e560ea None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:00.916962 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373faf73-ccb6-4dc5-aa99-c9a4cdb008d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:00.917649 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373faf73-ccb6-4dc5-aa99-c9a4cdb008d9 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:00.917935 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373faf73-ccb6-4dc5-aa99-c9a4cdb008d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:00.918211 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373faf73-ccb6-4dc5-aa99-c9a4cdb008d9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:00.923140 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-c40f4faf-abc9-432b-95a6-f0bdc4e560ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:32:00.923140 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:32:00.923140 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:32:00.923140 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:32:00.923140 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:32:00.923140 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:32:00.923140 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:32:00.923140 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:32:00.923140 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:32:00.923140 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:32:00.923140 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:32:00.923140 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:32:00.923140 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:32:00.923140 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:32:00.923140 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:32:00.924030 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:32:00.924030 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:32:00.924030 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:32:00.924030 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:32:00.924030 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:32:00.924030 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:32:00.924030 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:32:00.924030 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:32:00.924030 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( Jul 28 17:32:00.924030 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:32:00.924030 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:32:00.924030 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:32:00.924030 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:32:00.924030 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:32:00.924030 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:32:00.924030 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:32:00.924030 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:32:00.924946 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:32:00.924946 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:32:00.924946 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:32:00.924946 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:32:00.925643 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 425/828] 10.222.0.89 () {56 vars in 1108 bytes} [Thu Jul 28 17:32:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:32:00.926556 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40f4faf-abc9-432b-95a6-f0bdc4e560ea None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:00.932073 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373faf73-ccb6-4dc5-aa99-c9a4cdb008d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:32:00.938593 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-373faf73-ccb6-4dc5-aa99-c9a4cdb008d9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:32:00.938593 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:32:00.938593 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:32:00.938593 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:32:00.938593 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:32:00.938593 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:32:00.938593 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:32:00.938593 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:32:00.938593 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:32:00.938593 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:32:00.938593 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:32:00.938593 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:32:00.938593 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:32:00.938593 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:32:00.938593 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:32:00.938593 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:32:00.939558 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:32:00.939558 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:32:00.939558 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:32:00.939558 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:32:00.939558 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:32:00.939558 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:32:00.939558 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:32:00.939558 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:32:00.939558 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:32:00.939558 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:32:00.939558 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:32:00.939558 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:32:00.939558 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:32:00.939558 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:32:00.939558 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:32:00.939558 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:32:00.940419 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:32:00.940419 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:32:00.940419 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:32:00.940419 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:32:00.940419 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:32:00.940419 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:32:00.940419 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:32:00.940419 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:32:00.940419 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:32:00.940419 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:32:00.940419 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:32:00.940419 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:32:00.942503 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 404/829] 10.222.0.89 () {56 vars in 1096 bytes} [Thu Jul 28 17:32:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:32:00.945771 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373faf73-ccb6-4dc5-aa99-c9a4cdb008d9 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:00.965924 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:00.968197 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:01.066064 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40f4faf-abc9-432b-95a6-f0bdc4e560ea None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:01.070339 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373faf73-ccb6-4dc5-aa99-c9a4cdb008d9 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:01.105529 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea7f481-baa8-4c31-8fba-4592a1618fb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:01.106154 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea7f481-baa8-4c31-8fba-4592a1618fb3 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:01.106437 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea7f481-baa8-4c31-8fba-4592a1618fb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:01.106711 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea7f481-baa8-4c31-8fba-4592a1618fb3 None admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:01.107551 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea7f481-baa8-4c31-8fba-4592a1618fb3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:32:01.107655 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33427474-6cd0-4faf-89db-da7bde715747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:01.107872 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea7f481-baa8-4c31-8fba-4592a1618fb3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:01.108364 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33427474-6cd0-4faf-89db-da7bde715747 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:01.108642 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33427474-6cd0-4faf-89db-da7bde715747 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:01.108910 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33427474-6cd0-4faf-89db-da7bde715747 None admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:01.109437 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea7f481-baa8-4c31-8fba-4592a1618fb3 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:01.109638 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33427474-6cd0-4faf-89db-da7bde715747 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:32:01.109941 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33427474-6cd0-4faf-89db-da7bde715747 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:01.111287 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33427474-6cd0-4faf-89db-da7bde715747 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:01.128153 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 426/830] 10.222.0.89 () {56 vars in 1148 bytes} [Thu Jul 28 17:32:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:32:01.128153 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea7f481-baa8-4c31-8fba-4592a1618fb3 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:01.132713 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 405/831] 10.222.0.89 () {56 vars in 1109 bytes} [Thu Jul 28 17:32:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:32:01.133251 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33427474-6cd0-4faf-89db-da7bde715747 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:01.202814 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:01.220041 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:01.235473 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea7f481-baa8-4c31-8fba-4592a1618fb3 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:01.237799 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33427474-6cd0-4faf-89db-da7bde715747 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:01.269702 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0513eac2-cbc5-4aca-b811-26563fa9539e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:01.270315 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0513eac2-cbc5-4aca-b811-26563fa9539e None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:01.270599 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0513eac2-cbc5-4aca-b811-26563fa9539e None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:01.270878 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0513eac2-cbc5-4aca-b811-26563fa9539e None admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:01.271357 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24ff762-fd6e-4bc0-bbdf-c4d9fff94e5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:01.271595 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0513eac2-cbc5-4aca-b811-26563fa9539e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:32:01.271902 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0513eac2-cbc5-4aca-b811-26563fa9539e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:01.272024 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24ff762-fd6e-4bc0-bbdf-c4d9fff94e5f None admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:01.272301 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24ff762-fd6e-4bc0-bbdf-c4d9fff94e5f None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:01.272573 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24ff762-fd6e-4bc0-bbdf-c4d9fff94e5f None admin] PATH_INFO: `/v3/projects/04434e8535a64044996316f1d17dc0bc/users/c44387d13a694ee6bb47a13906003330/roles/0a9ea6e6d6f94955a5dc949b0746a3f1` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:01.273427 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0513eac2-cbc5-4aca-b811-26563fa9539e None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:01.282237 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 427/832] 10.222.0.89 () {56 vars in 1099 bytes} [Thu Jul 28 17:32:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:32:01.283226 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0513eac2-cbc5-4aca-b811-26563fa9539e None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:01.287407 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24ff762-fd6e-4bc0-bbdf-c4d9fff94e5f None admin] RBAC: Authorizing `identity:create_grant(project_id=04434e8535a64044996316f1d17dc0bc, user_id=c44387d13a694ee6bb47a13906003330, role_id=0a9ea6e6d6f94955a5dc949b0746a3f1)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:01.288839 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24ff762-fd6e-4bc0-bbdf-c4d9fff94e5f None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:01.302312 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 406/833] 10.222.0.89 () {58 vars in 1439 bytes} [Thu Jul 28 17:32:01 2022] PUT /identity/v3/projects/04434e8535a64044996316f1d17dc0bc/users/c44387d13a694ee6bb47a13906003330/roles/0a9ea6e6d6f94955a5dc949b0746a3f1 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:32:01.303622 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469f1d22-2d2b-45e3-93e2-56870e562505 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:01.303983 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469f1d22-2d2b-45e3-93e2-56870e562505 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:01.304361 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469f1d22-2d2b-45e3-93e2-56870e562505 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:01.305718 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 407/834] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:32:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:32:01.306552 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469f1d22-2d2b-45e3-93e2-56870e562505 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:01.416892 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0513eac2-cbc5-4aca-b811-26563fa9539e None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:01.426453 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469f1d22-2d2b-45e3-93e2-56870e562505 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:01.446632 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:01.465433 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab5ab8c-da80-4ebd-8178-1522cb1708c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:01.466105 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab5ab8c-da80-4ebd-8178-1522cb1708c3 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:01.466413 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab5ab8c-da80-4ebd-8178-1522cb1708c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:01.466689 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab5ab8c-da80-4ebd-8178-1522cb1708c3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:01.473524 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b884de7-ae06-4b2a-8c1f-698b4466e8af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:01.474390 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b884de7-ae06-4b2a-8c1f-698b4466e8af None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:01.474674 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b884de7-ae06-4b2a-8c1f-698b4466e8af None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:01.474946 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b884de7-ae06-4b2a-8c1f-698b4466e8af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:01.475068 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab5ab8c-da80-4ebd-8178-1522cb1708c3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:01.476388 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab5ab8c-da80-4ebd-8178-1522cb1708c3 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:01.486809 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b884de7-ae06-4b2a-8c1f-698b4466e8af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:01.488269 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b884de7-ae06-4b2a-8c1f-698b4466e8af None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:01.488371 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-0ab5ab8c-da80-4ebd-8178-1522cb1708c3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:32:01.488371 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:32:01.488371 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:32:01.488371 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:32:01.488371 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:32:01.488371 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:32:01.488371 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:32:01.488371 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:32:01.488371 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:32:01.488371 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:32:01.488371 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:32:01.488371 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:32:01.488371 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 17:32:01.488371 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 17:32:01.488371 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 17:32:01.489457 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 17:32:01.489457 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:32:01.489457 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:32:01.489457 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:32:01.489457 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:32:01.489457 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:32:01.489457 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:32:01.489457 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:32:01.489457 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( Jul 28 17:32:01.489457 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:32:01.489457 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:32:01.489457 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:32:01.489457 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:32:01.489457 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:32:01.489457 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:32:01.489457 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:32:01.489457 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:32:01.490890 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 17:32:01.490890 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 17:32:01.490890 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 17:32:01.490890 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 17:32:01.490890 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 17:32:01.490890 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 17:32:01.490890 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:32:01.490890 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:32:01.490890 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 428/835] 10.222.0.89 () {56 vars in 1111 bytes} [Thu Jul 28 17:32:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:32:01.490890 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85624924-70e0-4038-a6be-605965131c3b None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:01.491438 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85624924-70e0-4038-a6be-605965131c3b None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:01.491438 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85624924-70e0-4038-a6be-605965131c3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:01.506043 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-0b884de7-ae06-4b2a-8c1f-698b4466e8af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:32:01.506043 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:32:01.506043 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:32:01.506043 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:32:01.506043 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:32:01.506043 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:32:01.506043 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:32:01.506043 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:32:01.506043 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:32:01.506043 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:32:01.506043 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:32:01.506043 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:32:01.506043 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:32:01.506043 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:32:01.506043 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:32:01.507030 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:32:01.507030 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:32:01.507030 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:32:01.507030 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:32:01.507030 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:32:01.507030 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:32:01.507030 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:32:01.507030 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:32:01.507030 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:32:01.507030 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:32:01.507030 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:32:01.507030 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:32:01.507030 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:32:01.507030 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:32:01.507030 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:32:01.507030 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:32:01.507030 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:32:01.509032 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:32:01.509032 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:32:01.509032 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:32:01.509032 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:32:01.509032 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 408/836] 10.222.0.89 () {56 vars in 1108 bytes} [Thu Jul 28 17:32:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:32:01.509032 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b884de7-ae06-4b2a-8c1f-698b4466e8af None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:01.516109 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-85624924-70e0-4038-a6be-605965131c3b None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:32:01.517353 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-85624924-70e0-4038-a6be-605965131c3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:32:01.519517 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:01.589143 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 429/837] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:32:01 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:32:01.595236 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85624924-70e0-4038-a6be-605965131c3b None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:01.621531 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b884de7-ae06-4b2a-8c1f-698b4466e8af None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:01.657482 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464ed65e-d167-437f-9e46-087912ea248a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:01.658109 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464ed65e-d167-437f-9e46-087912ea248a None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:01.658388 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464ed65e-d167-437f-9e46-087912ea248a None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:01.658665 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464ed65e-d167-437f-9e46-087912ea248a None admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:01.659452 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464ed65e-d167-437f-9e46-087912ea248a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:32:01.659761 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464ed65e-d167-437f-9e46-087912ea248a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:01.661207 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464ed65e-d167-437f-9e46-087912ea248a None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:01.675433 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 409/838] 10.222.0.89 () {56 vars in 1148 bytes} [Thu Jul 28 17:32:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:32:01.676461 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464ed65e-d167-437f-9e46-087912ea248a None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:01.709274 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:01.718712 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85624924-70e0-4038-a6be-605965131c3b None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:01.757641 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db5ef0f-1f86-4633-9adc-72396fd81ac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:01.758255 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db5ef0f-1f86-4633-9adc-72396fd81ac9 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:01.758532 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db5ef0f-1f86-4633-9adc-72396fd81ac9 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:01.758800 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db5ef0f-1f86-4633-9adc-72396fd81ac9 None admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:01.759482 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db5ef0f-1f86-4633-9adc-72396fd81ac9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:32:01.759780 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db5ef0f-1f86-4633-9adc-72396fd81ac9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:01.761186 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db5ef0f-1f86-4633-9adc-72396fd81ac9 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:01.769748 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:01.777900 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 430/839] 10.222.0.89 () {56 vars in 1109 bytes} [Thu Jul 28 17:32:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:32:01.778826 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db5ef0f-1f86-4633-9adc-72396fd81ac9 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:01.794639 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464ed65e-d167-437f-9e46-087912ea248a None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:01.836793 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4943c751-b498-46fa-a7ee-7dbd8b35cb83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:01.837477 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4943c751-b498-46fa-a7ee-7dbd8b35cb83 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:01.837817 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4943c751-b498-46fa-a7ee-7dbd8b35cb83 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:01.838095 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4943c751-b498-46fa-a7ee-7dbd8b35cb83 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:01.838736 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4943c751-b498-46fa-a7ee-7dbd8b35cb83 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:01.840091 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4943c751-b498-46fa-a7ee-7dbd8b35cb83 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:01.850370 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-4943c751-b498-46fa-a7ee-7dbd8b35cb83 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 17:32:01.850370 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:32:01.850370 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 17:32:01.850370 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 17:32:01.850370 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 17:32:01.850370 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 17:32:01.850370 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 17:32:01.850370 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:32:01.850370 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 17:32:01.850370 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:32:01.850370 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 17:32:01.850370 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:32:01.850370 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:32:01.850370 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:32:01.850370 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:32:01.850370 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:32:01.850370 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:32:01.851381 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:32:01.851381 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:32:01.851381 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:32:01.851381 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:32:01.851381 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:32:01.851381 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:32:01.851381 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 17:32:01.851381 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 17:32:01.851381 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 17:32:01.851381 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 17:32:01.851381 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:32:01.851381 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:32:01.851381 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:32:01.851381 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:32:01.851381 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:32:01.851381 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:32:01.852225 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:32:01.852225 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:32:01.852225 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:32:01.852225 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:32:01.852225 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:32:01.852225 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:32:01.852225 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:32:01.852225 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:32:01.852225 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:32:01.852225 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:32:01.852225 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 17:32:01.852225 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:32:01.852225 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 17:32:01.852225 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:32:01.852225 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 410/840] 10.222.0.89 () {56 vars in 1114 bytes} [Thu Jul 28 17:32:01 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:32:01.853094 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4943c751-b498-46fa-a7ee-7dbd8b35cb83 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:01.897010 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db5ef0f-1f86-4633-9adc-72396fd81ac9 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:01.932178 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b92143-2021-4a68-99ee-9af9c0a368c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:01.932798 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b92143-2021-4a68-99ee-9af9c0a368c7 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:01.933080 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b92143-2021-4a68-99ee-9af9c0a368c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:01.933385 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b92143-2021-4a68-99ee-9af9c0a368c7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:01.934171 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b92143-2021-4a68-99ee-9af9c0a368c7 None admin] RBAC: Adding query filter params (role.id=73e457231a824ea98c28d22610be640c, scope.project.id=04434e8535a64044996316f1d17dc0bc, user.id=d10b49b29df94404a71f43273839c252) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:32:01.934485 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b92143-2021-4a68-99ee-9af9c0a368c7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:01.936566 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b92143-2021-4a68-99ee-9af9c0a368c7 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:01.942795 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 431/841] 10.222.0.89 () {56 vars in 1374 bytes} [Thu Jul 28 17:32:01 2022] GET /identity/v3/role_assignments?user.id=d10b49b29df94404a71f43273839c252&scope.project.id=04434e8535a64044996316f1d17dc0bc&role.id=73e457231a824ea98c28d22610be640c => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:32:01.943588 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b92143-2021-4a68-99ee-9af9c0a368c7 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:01.971371 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:01.983053 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4943c751-b498-46fa-a7ee-7dbd8b35cb83 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:02.025224 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:02.027218 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b07eb2-2310-4908-8af7-bfd0bacb5e13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:02.027958 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b07eb2-2310-4908-8af7-bfd0bacb5e13 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:02.028407 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b07eb2-2310-4908-8af7-bfd0bacb5e13 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:02.028692 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b07eb2-2310-4908-8af7-bfd0bacb5e13 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:02.038201 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b07eb2-2310-4908-8af7-bfd0bacb5e13 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:02.039684 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b07eb2-2310-4908-8af7-bfd0bacb5e13 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:02.054312 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-80b07eb2-2310-4908-8af7-bfd0bacb5e13 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 28 17:32:02.054312 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:32:02.054312 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:32:02.054312 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:32:02.054312 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:32:02.054312 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:32:02.054312 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:32:02.054312 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:32:02.054312 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:32:02.054312 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:32:02.054312 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:32:02.054312 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:32:02.054312 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 17:32:02.054312 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 17:32:02.054312 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 17:32:02.054312 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 17:32:02.055337 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:32:02.055337 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:32:02.055337 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 17:32:02.055337 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:32:02.055337 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 17:32:02.055337 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:32:02.055337 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:32:02.055337 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:32:02.055337 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:32:02.055337 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:32:02.055337 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:32:02.055337 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:32:02.055337 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:32:02.055337 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:32:02.055337 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:32:02.055337 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:32:02.055337 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:32:02.055337 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:32:02.056295 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:32:02.056295 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:32:02.056295 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 17:32:02.056295 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 17:32:02.056295 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 17:32:02.056295 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 17:32:02.056295 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 17:32:02.056295 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 17:32:02.056295 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 28 17:32:02.056295 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:32:02.061333 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 411/842] 10.222.0.89 () {56 vars in 1096 bytes} [Thu Jul 28 17:32:01 2022] GET /identity/v3/users/swift => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:32:02.062773 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b07eb2-2310-4908-8af7-bfd0bacb5e13 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:02.086502 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b92143-2021-4a68-99ee-9af9c0a368c7 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:02.138229 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f481c45-4398-4392-b32f-42a8d4330009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:02.138995 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f481c45-4398-4392-b32f-42a8d4330009 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:02.139348 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f481c45-4398-4392-b32f-42a8d4330009 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:02.139633 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f481c45-4398-4392-b32f-42a8d4330009 None admin] PATH_INFO: `/v3/services` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:02.140331 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f481c45-4398-4392-b32f-42a8d4330009 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:32:02.140666 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f481c45-4398-4392-b32f-42a8d4330009 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:02.142111 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f481c45-4398-4392-b32f-42a8d4330009 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:02.147950 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 432/843] 10.222.0.89 () {56 vars in 1114 bytes} [Thu Jul 28 17:32:01 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:32:02.154160 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f481c45-4398-4392-b32f-42a8d4330009 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:02.178929 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b07eb2-2310-4908-8af7-bfd0bacb5e13 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:02.223273 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735a7c0a-b025-4167-a759-11dd007ef209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:02.223969 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735a7c0a-b025-4167-a759-11dd007ef209 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:02.224366 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735a7c0a-b025-4167-a759-11dd007ef209 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:02.224728 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735a7c0a-b025-4167-a759-11dd007ef209 None admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:02.225665 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735a7c0a-b025-4167-a759-11dd007ef209 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:32:02.226095 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735a7c0a-b025-4167-a759-11dd007ef209 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:02.227622 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735a7c0a-b025-4167-a759-11dd007ef209 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:02.247110 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 412/844] 10.222.0.89 () {56 vars in 1135 bytes} [Thu Jul 28 17:32:02 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:32:02.253224 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735a7c0a-b025-4167-a759-11dd007ef209 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:02.271449 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f481c45-4398-4392-b32f-42a8d4330009 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:02.278916 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:02.311840 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:02.324315 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc2e12a-24f3-4a26-a84b-f2b2f8672d13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:02.325226 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc2e12a-24f3-4a26-a84b-f2b2f8672d13 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:02.325870 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc2e12a-24f3-4a26-a84b-f2b2f8672d13 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:02.327378 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc2e12a-24f3-4a26-a84b-f2b2f8672d13 None admin] PATH_INFO: `/v3/services` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:02.328264 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc2e12a-24f3-4a26-a84b-f2b2f8672d13 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:32:02.328801 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc2e12a-24f3-4a26-a84b-f2b2f8672d13 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:02.330950 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc2e12a-24f3-4a26-a84b-f2b2f8672d13 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:02.342889 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 433/845] 10.222.0.89 () {56 vars in 1114 bytes} [Thu Jul 28 17:32:02 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:32:02.396665 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735a7c0a-b025-4167-a759-11dd007ef209 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:02.439166 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e67615-1944-4012-8007-9eb76350b564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:02.439842 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e67615-1944-4012-8007-9eb76350b564 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:02.440143 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e67615-1944-4012-8007-9eb76350b564 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:02.440439 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e67615-1944-4012-8007-9eb76350b564 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:02.530112 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e67615-1944-4012-8007-9eb76350b564 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:02.531539 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e67615-1944-4012-8007-9eb76350b564 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:02.546973 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:02.641396 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 413/846] 10.222.0.89 () {58 vars in 1282 bytes} [Thu Jul 28 17:32:02 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 17:32:02.646686 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc2e12a-24f3-4a26-a84b-f2b2f8672d13 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:02.759961 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc2e12a-24f3-4a26-a84b-f2b2f8672d13 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:02.797211 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e123875-bdf0-4624-a31e-d774ad7740ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:02.797770 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e123875-bdf0-4624-a31e-d774ad7740ae None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:02.798050 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e123875-bdf0-4624-a31e-d774ad7740ae None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:02.798319 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e123875-bdf0-4624-a31e-d774ad7740ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:02.810359 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e123875-bdf0-4624-a31e-d774ad7740ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:02.812832 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e123875-bdf0-4624-a31e-d774ad7740ae None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:02.829459 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-0e123875-bdf0-4624-a31e-d774ad7740ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:32:02.829459 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:32:02.829459 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:32:02.829459 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:32:02.829459 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:32:02.829459 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:32:02.829459 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:32:02.829459 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:32:02.829459 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:32:02.829459 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:32:02.829459 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:32:02.829459 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:32:02.829459 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:32:02.829459 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:32:02.829459 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:32:02.830432 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:32:02.830432 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:32:02.830432 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:32:02.830432 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:32:02.830432 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:32:02.830432 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:32:02.830432 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:32:02.830432 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:32:02.830432 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( Jul 28 17:32:02.830432 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:32:02.830432 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:32:02.830432 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:32:02.830432 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:32:02.830432 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:32:02.830432 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:32:02.830432 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:32:02.830432 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:32:02.831347 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:32:02.831347 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:32:02.831347 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:32:02.831347 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:32:02.831347 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 434/847] 10.222.0.89 () {56 vars in 1108 bytes} [Thu Jul 28 17:32:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:32:02.836824 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e67615-1944-4012-8007-9eb76350b564 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:02.943232 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e67615-1944-4012-8007-9eb76350b564 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:02.957422 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:02.979210 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f41f7e3-de74-46f0-94f2-b6d398f1665f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:02.979832 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f41f7e3-de74-46f0-94f2-b6d398f1665f None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:02.980114 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f41f7e3-de74-46f0-94f2-b6d398f1665f None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:02.980390 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f41f7e3-de74-46f0-94f2-b6d398f1665f None admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:02.981085 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f41f7e3-de74-46f0-94f2-b6d398f1665f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:32:02.981417 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f41f7e3-de74-46f0-94f2-b6d398f1665f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:02.983401 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f41f7e3-de74-46f0-94f2-b6d398f1665f None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:02.995875 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 414/848] 10.222.0.89 () {56 vars in 1109 bytes} [Thu Jul 28 17:32:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:32:03.002691 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e123875-bdf0-4624-a31e-d774ad7740ae None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:03.053632 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:03.109114 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e123875-bdf0-4624-a31e-d774ad7740ae None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:03.144014 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f8ab42-1e86-428f-b75a-7cdf322f16c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:03.144667 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f8ab42-1e86-428f-b75a-7cdf322f16c3 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:03.144945 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f8ab42-1e86-428f-b75a-7cdf322f16c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:03.145247 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f8ab42-1e86-428f-b75a-7cdf322f16c3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:03.152287 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f8ab42-1e86-428f-b75a-7cdf322f16c3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:03.153764 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f8ab42-1e86-428f-b75a-7cdf322f16c3 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:03.166848 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-93f8ab42-1e86-428f-b75a-7cdf322f16c3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:32:03.166848 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:32:03.166848 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:32:03.166848 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:32:03.166848 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:32:03.166848 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:32:03.166848 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:32:03.166848 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:32:03.166848 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:32:03.166848 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:32:03.166848 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:32:03.166848 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:32:03.166848 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 17:32:03.166848 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 17:32:03.166848 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 17:32:03.168900 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 17:32:03.168900 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:32:03.168900 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:32:03.168900 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:32:03.168900 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:32:03.168900 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:32:03.168900 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:32:03.168900 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:32:03.168900 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( Jul 28 17:32:03.168900 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:32:03.168900 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:32:03.168900 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:32:03.168900 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:32:03.168900 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:32:03.168900 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:32:03.168900 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:32:03.168900 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:32:03.170585 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 17:32:03.170585 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 17:32:03.170585 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 17:32:03.170585 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 17:32:03.170585 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 17:32:03.170585 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 17:32:03.170585 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:32:03.170585 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:32:03.170585 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 435/849] 10.222.0.89 () {56 vars in 1111 bytes} [Thu Jul 28 17:32:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:32:03.175598 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f41f7e3-de74-46f0-94f2-b6d398f1665f None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:03.289544 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f41f7e3-de74-46f0-94f2-b6d398f1665f None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:03.325516 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdb3c96-aeed-4a23-8260-a82d36321abb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:03.326731 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdb3c96-aeed-4a23-8260-a82d36321abb None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:03.327093 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdb3c96-aeed-4a23-8260-a82d36321abb None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:03.327429 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdb3c96-aeed-4a23-8260-a82d36321abb None admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:03.328197 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdb3c96-aeed-4a23-8260-a82d36321abb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:32:03.328504 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdb3c96-aeed-4a23-8260-a82d36321abb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:03.330014 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdb3c96-aeed-4a23-8260-a82d36321abb None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:03.342963 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 415/850] 10.222.0.89 () {56 vars in 1148 bytes} [Thu Jul 28 17:32:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:32:03.348560 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f8ab42-1e86-428f-b75a-7cdf322f16c3 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:03.423968 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab93224e-cd20-4236-ab6d-f77030c8aa06 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:03.424284 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab93224e-cd20-4236-ab6d-f77030c8aa06 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:03.424532 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab93224e-cd20-4236-ab6d-f77030c8aa06 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:03.427131 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 416/851] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:32:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 17:32:03.431014 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d678d6a-3c6f-421c-9b95-42e8387e34f3 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:03.431301 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d678d6a-3c6f-421c-9b95-42e8387e34f3 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:03.431581 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d678d6a-3c6f-421c-9b95-42e8387e34f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:03.453765 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-4d678d6a-3c6f-421c-9b95-42e8387e34f3 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:32:03.455013 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-4d678d6a-3c6f-421c-9b95-42e8387e34f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:32:03.462231 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f8ab42-1e86-428f-b75a-7cdf322f16c3 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:03.470956 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:03.491649 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:03.503978 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403f42c9-4e68-4813-bd0f-98c44d2f1db5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:03.505223 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403f42c9-4e68-4813-bd0f-98c44d2f1db5 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:03.505710 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403f42c9-4e68-4813-bd0f-98c44d2f1db5 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:03.506179 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403f42c9-4e68-4813-bd0f-98c44d2f1db5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:03.507171 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403f42c9-4e68-4813-bd0f-98c44d2f1db5 None admin] RBAC: Adding query filter params (role.id=73e457231a824ea98c28d22610be640c, scope.project.id=04434e8535a64044996316f1d17dc0bc, user.id=3c59473b01bf4433ba3b111114fc048b) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:32:03.507682 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403f42c9-4e68-4813-bd0f-98c44d2f1db5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:03.509378 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403f42c9-4e68-4813-bd0f-98c44d2f1db5 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:03.517021 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 436/852] 10.222.0.89 () {56 vars in 1374 bytes} [Thu Jul 28 17:32:03 2022] GET /identity/v3/role_assignments?user.id=3c59473b01bf4433ba3b111114fc048b&scope.project.id=04434e8535a64044996316f1d17dc0bc&role.id=73e457231a824ea98c28d22610be640c => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:32:03.549283 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 417/853] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:32:03 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 17:32:03.567081 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403f42c9-4e68-4813-bd0f-98c44d2f1db5 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:03.680381 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403f42c9-4e68-4813-bd0f-98c44d2f1db5 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:03.718345 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a226c4e6-b6ae-49e5-8e97-68c8d910b68a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:03.718990 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a226c4e6-b6ae-49e5-8e97-68c8d910b68a None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:03.719269 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a226c4e6-b6ae-49e5-8e97-68c8d910b68a None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:03.719544 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a226c4e6-b6ae-49e5-8e97-68c8d910b68a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:03.730240 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a226c4e6-b6ae-49e5-8e97-68c8d910b68a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:32:03.735693 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-a226c4e6-b6ae-49e5-8e97-68c8d910b68a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 17:32:03.735693 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:32:03.735693 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:32:03.735693 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:32:03.735693 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:32:03.735693 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:32:03.735693 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:32:03.735693 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:32:03.735693 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:32:03.735693 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:32:03.735693 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:32:03.735693 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:32:03.735693 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:32:03.735693 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:32:03.735693 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:32:03.735693 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:32:03.736737 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:32:03.736737 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:32:03.736737 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:32:03.736737 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:32:03.736737 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:32:03.736737 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:32:03.736737 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:32:03.736737 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:32:03.736737 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:32:03.736737 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:32:03.736737 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:32:03.736737 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( Jul 28 17:32:03.736737 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:32:03.736737 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:32:03.736737 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:32:03.736737 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:32:03.737823 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:32:03.737823 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:32:03.737823 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:32:03.737823 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:32:03.737823 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:32:03.737823 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:32:03.737823 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:32:03.737823 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:32:03.737823 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:32:03.737823 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:32:03.737823 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 17:32:03.737823 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:32:03.737823 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 437/854] 10.222.0.89 () {56 vars in 1102 bytes} [Thu Jul 28 17:32:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:32:03.743245 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d678d6a-3c6f-421c-9b95-42e8387e34f3 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:03.851138 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d678d6a-3c6f-421c-9b95-42e8387e34f3 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:03.889802 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5f94d9-6353-4cdc-a2a4-53acd9cbc027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:03.890421 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5f94d9-6353-4cdc-a2a4-53acd9cbc027 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:03.890848 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5f94d9-6353-4cdc-a2a4-53acd9cbc027 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:03.891264 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5f94d9-6353-4cdc-a2a4-53acd9cbc027 None admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:03.892379 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5f94d9-6353-4cdc-a2a4-53acd9cbc027 None admin] RBAC: Adding query filter params (name=service) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:32:03.892852 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5f94d9-6353-4cdc-a2a4-53acd9cbc027 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:03.895061 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5f94d9-6353-4cdc-a2a4-53acd9cbc027 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:03.906442 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 418/855] 10.222.0.89 () {56 vars in 1103 bytes} [Thu Jul 28 17:32:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:32:03.912283 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a226c4e6-b6ae-49e5-8e97-68c8d910b68a None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:03.944154 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:04.027203 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a226c4e6-b6ae-49e5-8e97-68c8d910b68a None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:04.065937 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa2d6c6-7262-4b5c-b9bf-653a04ef5e3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:04.066944 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa2d6c6-7262-4b5c-b9bf-653a04ef5e3f None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:04.067492 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa2d6c6-7262-4b5c-b9bf-653a04ef5e3f None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:04.067963 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa2d6c6-7262-4b5c-b9bf-653a04ef5e3f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:04.080536 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa2d6c6-7262-4b5c-b9bf-653a04ef5e3f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:04.082578 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa2d6c6-7262-4b5c-b9bf-653a04ef5e3f None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:04.098996 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-5aa2d6c6-7262-4b5c-b9bf-653a04ef5e3f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:32:04.098996 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:32:04.098996 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:32:04.098996 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:32:04.098996 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:32:04.098996 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:32:04.098996 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:32:04.098996 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:32:04.098996 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:32:04.098996 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:32:04.098996 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:32:04.098996 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:32:04.098996 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:32:04.098996 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:32:04.098996 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:32:04.099935 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:32:04.099935 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:32:04.099935 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:32:04.099935 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:32:04.099935 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:32:04.099935 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:32:04.099935 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:32:04.099935 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:32:04.099935 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( Jul 28 17:32:04.099935 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:32:04.099935 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:32:04.099935 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:32:04.099935 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:32:04.099935 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:32:04.099935 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:32:04.099935 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:32:04.099935 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:32:04.100821 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:32:04.100821 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:32:04.100821 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:32:04.100821 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:32:04.103009 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 438/856] 10.222.0.89 () {56 vars in 1108 bytes} [Thu Jul 28 17:32:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:32:04.111463 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5f94d9-6353-4cdc-a2a4-53acd9cbc027 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:04.140148 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:04.221008 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa4aaf3-a7be-42d5-8e72-6056d0245c81 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:04.221529 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa4aaf3-a7be-42d5-8e72-6056d0245c81 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:04.221529 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa4aaf3-a7be-42d5-8e72-6056d0245c81 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:04.222972 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 439/857] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:32:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 17:32:04.227709 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279185c8-ab16-40eb-af9e-5d554ad3dd7d None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:04.227998 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279185c8-ab16-40eb-af9e-5d554ad3dd7d None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:04.228303 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279185c8-ab16-40eb-af9e-5d554ad3dd7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:04.228567 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5f94d9-6353-4cdc-a2a4-53acd9cbc027 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:04.249375 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-279185c8-ab16-40eb-af9e-5d554ad3dd7d None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:32:04.250596 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-279185c8-ab16-40eb-af9e-5d554ad3dd7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:32:04.266090 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e10a3f-66b6-4146-bcb3-959c4e6b641c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:04.267140 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e10a3f-66b6-4146-bcb3-959c4e6b641c None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:04.267713 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e10a3f-66b6-4146-bcb3-959c4e6b641c None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:04.268183 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e10a3f-66b6-4146-bcb3-959c4e6b641c None admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:04.269077 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e10a3f-66b6-4146-bcb3-959c4e6b641c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:32:04.269610 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e10a3f-66b6-4146-bcb3-959c4e6b641c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:04.271348 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e10a3f-66b6-4146-bcb3-959c4e6b641c None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:04.286771 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 419/858] 10.222.0.89 () {56 vars in 1109 bytes} [Thu Jul 28 17:32:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:32:04.297211 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e10a3f-66b6-4146-bcb3-959c4e6b641c None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:04.331628 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 440/859] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:32:04 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 17:32:04.356569 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279185c8-ab16-40eb-af9e-5d554ad3dd7d None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:04.368212 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:04.400961 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:04.419984 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e10a3f-66b6-4146-bcb3-959c4e6b641c None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:04.457485 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d437487d-7922-4585-8676-9c7205eee2a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:04.458131 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d437487d-7922-4585-8676-9c7205eee2a7 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:04.458413 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d437487d-7922-4585-8676-9c7205eee2a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:04.458686 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d437487d-7922-4585-8676-9c7205eee2a7 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:04.465727 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d437487d-7922-4585-8676-9c7205eee2a7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:04.468025 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d437487d-7922-4585-8676-9c7205eee2a7 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:04.473230 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279185c8-ab16-40eb-af9e-5d554ad3dd7d None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:04.479569 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-d437487d-7922-4585-8676-9c7205eee2a7 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 17:32:04.479569 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:32:04.479569 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:32:04.479569 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:32:04.479569 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:32:04.479569 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:32:04.479569 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:32:04.479569 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:32:04.479569 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:32:04.479569 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:32:04.479569 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:32:04.479569 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:32:04.479569 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 17:32:04.479569 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 17:32:04.479569 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 17:32:04.479569 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 17:32:04.480556 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:32:04.480556 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:32:04.480556 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 17:32:04.480556 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:32:04.480556 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 17:32:04.480556 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:32:04.480556 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:32:04.480556 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:32:04.480556 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:32:04.480556 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:32:04.480556 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:32:04.480556 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:32:04.480556 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:32:04.480556 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:32:04.480556 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:32:04.480556 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:32:04.480556 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:32:04.480556 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:32:04.481586 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:32:04.481586 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:32:04.481586 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 17:32:04.481586 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 17:32:04.481586 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 17:32:04.481586 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 17:32:04.481586 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 17:32:04.481586 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 17:32:04.481586 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 17:32:04.481586 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:32:04.481586 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 420/860] 10.222.0.89 () {56 vars in 1117 bytes} [Thu Jul 28 17:32:04 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:32:04.482289 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b936d0f6-80c4-45bb-bd05-50f53cc19fb3 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:04.482289 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b936d0f6-80c4-45bb-bd05-50f53cc19fb3 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:04.482944 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b936d0f6-80c4-45bb-bd05-50f53cc19fb3 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:04.485288 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 421/861] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:32:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:32:04.486922 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b936d0f6-80c4-45bb-bd05-50f53cc19fb3 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:04.519273 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ab658e-adf0-4f12-a525-f15e2c599e63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:04.519905 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ab658e-adf0-4f12-a525-f15e2c599e63 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:04.520180 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ab658e-adf0-4f12-a525-f15e2c599e63 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:04.520522 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ab658e-adf0-4f12-a525-f15e2c599e63 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:04.521063 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ab658e-adf0-4f12-a525-f15e2c599e63 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:04.522447 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ab658e-adf0-4f12-a525-f15e2c599e63 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:04.533640 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-a4ab658e-adf0-4f12-a525-f15e2c599e63 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 17:32:04.533640 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:32:04.533640 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 17:32:04.533640 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 17:32:04.533640 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 17:32:04.533640 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 17:32:04.533640 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 17:32:04.533640 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:32:04.533640 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 17:32:04.533640 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:32:04.533640 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 17:32:04.533640 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:32:04.533640 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:32:04.533640 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:32:04.533640 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:32:04.533640 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:32:04.534603 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:32:04.534603 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:32:04.534603 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:32:04.534603 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:32:04.534603 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:32:04.534603 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:32:04.534603 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:32:04.534603 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 17:32:04.534603 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 17:32:04.534603 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 17:32:04.534603 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 17:32:04.534603 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:32:04.534603 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:32:04.534603 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:32:04.534603 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:32:04.534603 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:32:04.536572 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:32:04.536572 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:32:04.536572 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( Jul 28 17:32:04.536572 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:32:04.536572 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:32:04.536572 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:32:04.536572 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:32:04.536572 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:32:04.536572 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:32:04.536572 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:32:04.536572 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:32:04.536572 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 17:32:04.536572 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:32:04.536572 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 17:32:04.536572 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:32:04.536572 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 441/862] 10.222.0.89 () {56 vars in 1132 bytes} [Thu Jul 28 17:32:04 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:32:04.537507 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e02d05e-cf0b-4039-af49-bc78ba0112ff None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:04.537507 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e02d05e-cf0b-4039-af49-bc78ba0112ff None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:04.537507 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e02d05e-cf0b-4039-af49-bc78ba0112ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:04.555291 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-7e02d05e-cf0b-4039-af49-bc78ba0112ff None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:32:04.556623 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-7e02d05e-cf0b-4039-af49-bc78ba0112ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:32:04.605582 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b936d0f6-80c4-45bb-bd05-50f53cc19fb3 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:04.634833 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 442/863] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:32:04 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:32:04.636038 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e02d05e-cf0b-4039-af49-bc78ba0112ff None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:04.648191 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:04.653821 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd9d12a-b85b-41eb-94b0-8db843d44c53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:04.655784 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd9d12a-b85b-41eb-94b0-8db843d44c53 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:04.656230 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd9d12a-b85b-41eb-94b0-8db843d44c53 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:04.656503 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd9d12a-b85b-41eb-94b0-8db843d44c53 None admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:04.657534 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd9d12a-b85b-41eb-94b0-8db843d44c53 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:32:04.657839 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd9d12a-b85b-41eb-94b0-8db843d44c53 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:04.659320 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd9d12a-b85b-41eb-94b0-8db843d44c53 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:04.677614 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 422/864] 10.222.0.89 () {56 vars in 1149 bytes} [Thu Jul 28 17:32:04 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:32:04.682845 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd9d12a-b85b-41eb-94b0-8db843d44c53 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:04.686036 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:04.764744 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e02d05e-cf0b-4039-af49-bc78ba0112ff None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:04.795443 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd9d12a-b85b-41eb-94b0-8db843d44c53 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:04.804110 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b23f47b-e2eb-4545-8af0-96a43b92ba21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:04.805054 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b23f47b-e2eb-4545-8af0-96a43b92ba21 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:04.805538 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b23f47b-e2eb-4545-8af0-96a43b92ba21 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:04.806119 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b23f47b-e2eb-4545-8af0-96a43b92ba21 None admin] PATH_INFO: `/v3/services` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:04.806966 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b23f47b-e2eb-4545-8af0-96a43b92ba21 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:32:04.807271 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b23f47b-e2eb-4545-8af0-96a43b92ba21 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:04.808655 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b23f47b-e2eb-4545-8af0-96a43b92ba21 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:04.816083 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 443/865] 10.222.0.89 () {56 vars in 1126 bytes} [Thu Jul 28 17:32:04 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:32:04.817276 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b23f47b-e2eb-4545-8af0-96a43b92ba21 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:04.838336 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20a15c1-743c-4f19-9bd6-2371aa02a8ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:04.839123 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20a15c1-743c-4f19-9bd6-2371aa02a8ca None admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:04.839378 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20a15c1-743c-4f19-9bd6-2371aa02a8ca None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:04.839758 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20a15c1-743c-4f19-9bd6-2371aa02a8ca None admin] PATH_INFO: `/v3/services` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:04.840493 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20a15c1-743c-4f19-9bd6-2371aa02a8ca None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:04.842389 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20a15c1-743c-4f19-9bd6-2371aa02a8ca None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:04.869188 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 423/866] 10.222.0.89 () {60 vars in 1141 bytes} [Thu Jul 28 17:32:04 2022] POST /identity/v3/services => generated 251 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:32:04.869188 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20a15c1-743c-4f19-9bd6-2371aa02a8ca None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:04.908074 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:04.948753 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:04.968561 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b23f47b-e2eb-4545-8af0-96a43b92ba21 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:05.007949 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20a15c1-743c-4f19-9bd6-2371aa02a8ca None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:05.025461 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7ddeb3-e2ae-4082-8c02-cb723fb5023d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:05.026161 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7ddeb3-e2ae-4082-8c02-cb723fb5023d None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:05.026563 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7ddeb3-e2ae-4082-8c02-cb723fb5023d None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:05.026848 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7ddeb3-e2ae-4082-8c02-cb723fb5023d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:05.051563 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee80859-fdd3-41ac-90d3-ed145f80faea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:05.052425 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee80859-fdd3-41ac-90d3-ed145f80faea None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:05.052849 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee80859-fdd3-41ac-90d3-ed145f80faea None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:05.053319 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee80859-fdd3-41ac-90d3-ed145f80faea None admin] PATH_INFO: `/v3/services` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:05.054164 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee80859-fdd3-41ac-90d3-ed145f80faea None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:32:05.054610 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee80859-fdd3-41ac-90d3-ed145f80faea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:05.056093 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee80859-fdd3-41ac-90d3-ed145f80faea None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:05.069271 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 424/867] 10.222.0.89 () {56 vars in 1126 bytes} [Thu Jul 28 17:32:04 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:32:05.121853 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7ddeb3-e2ae-4082-8c02-cb723fb5023d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:05.124455 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7ddeb3-e2ae-4082-8c02-cb723fb5023d None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:05.200855 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:05.250193 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 444/868] 10.222.0.89 () {58 vars in 1282 bytes} [Thu Jul 28 17:32:04 2022] GET /identity/v3/auth/tokens => generated 1252 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 17:32:05.256015 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee80859-fdd3-41ac-90d3-ed145f80faea None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:05.374016 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee80859-fdd3-41ac-90d3-ed145f80faea None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:05.398132 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:05.411077 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f89dde-0f88-4b8f-a046-c05031d2fc2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:05.411722 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f89dde-0f88-4b8f-a046-c05031d2fc2e None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:05.411998 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f89dde-0f88-4b8f-a046-c05031d2fc2e None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:05.412268 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f89dde-0f88-4b8f-a046-c05031d2fc2e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:05.423668 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f89dde-0f88-4b8f-a046-c05031d2fc2e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:05.425113 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f89dde-0f88-4b8f-a046-c05031d2fc2e None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:05.440829 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-d8f89dde-0f88-4b8f-a046-c05031d2fc2e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:32:05.440829 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:32:05.440829 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:32:05.440829 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:32:05.440829 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:32:05.440829 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:32:05.440829 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:32:05.440829 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:32:05.440829 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:32:05.440829 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:32:05.440829 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:32:05.440829 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:32:05.440829 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:32:05.440829 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:32:05.440829 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:32:05.441824 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:32:05.441824 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:32:05.441824 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:32:05.441824 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:32:05.441824 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:32:05.441824 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:32:05.441824 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:32:05.441824 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:32:05.441824 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:32:05.441824 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:32:05.441824 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:32:05.441824 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:32:05.441824 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:32:05.441824 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:32:05.441824 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:32:05.441824 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:32:05.441824 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:32:05.442785 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:32:05.442785 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:32:05.442785 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:32:05.442785 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:32:05.442785 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 425/869] 10.222.0.89 () {56 vars in 1108 bytes} [Thu Jul 28 17:32:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:32:05.449094 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7ddeb3-e2ae-4082-8c02-cb723fb5023d None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:05.561370 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7ddeb3-e2ae-4082-8c02-cb723fb5023d None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:05.598562 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e6c2b1-5fb7-41a6-bd81-68e29f2b344b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:05.599184 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e6c2b1-5fb7-41a6-bd81-68e29f2b344b None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:05.599462 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e6c2b1-5fb7-41a6-bd81-68e29f2b344b None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:05.599732 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e6c2b1-5fb7-41a6-bd81-68e29f2b344b None admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:05.600420 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e6c2b1-5fb7-41a6-bd81-68e29f2b344b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:32:05.600722 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e6c2b1-5fb7-41a6-bd81-68e29f2b344b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:05.602113 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e6c2b1-5fb7-41a6-bd81-68e29f2b344b None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:05.615380 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 445/870] 10.222.0.89 () {56 vars in 1109 bytes} [Thu Jul 28 17:32:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:32:05.620999 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f89dde-0f88-4b8f-a046-c05031d2fc2e None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:05.686051 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736fd48b-21b9-4332-99f9-09544086b909 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:05.686338 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736fd48b-21b9-4332-99f9-09544086b909 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:05.686666 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736fd48b-21b9-4332-99f9-09544086b909 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:05.688198 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 446/871] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:32:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 17:32:05.693026 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c74a632-2bb8-40be-a6ed-09ab0fa9177c None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:05.693341 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c74a632-2bb8-40be-a6ed-09ab0fa9177c None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:05.693615 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c74a632-2bb8-40be-a6ed-09ab0fa9177c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:05.713613 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-9c74a632-2bb8-40be-a6ed-09ab0fa9177c None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:32:05.714826 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-9c74a632-2bb8-40be-a6ed-09ab0fa9177c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:32:05.732914 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f89dde-0f88-4b8f-a046-c05031d2fc2e None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:05.737476 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:05.770758 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb6dcce-c873-4f58-b1a1-5c025fbfd506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:05.771584 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb6dcce-c873-4f58-b1a1-5c025fbfd506 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:05.771961 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb6dcce-c873-4f58-b1a1-5c025fbfd506 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:05.772337 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb6dcce-c873-4f58-b1a1-5c025fbfd506 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:05.781136 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb6dcce-c873-4f58-b1a1-5c025fbfd506 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:05.782832 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb6dcce-c873-4f58-b1a1-5c025fbfd506 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:05.796198 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-0eb6dcce-c873-4f58-b1a1-5c025fbfd506 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:32:05.796198 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:32:05.796198 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:32:05.796198 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:32:05.796198 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:32:05.796198 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:32:05.796198 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:32:05.796198 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:32:05.796198 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:32:05.796198 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:32:05.796198 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:32:05.796198 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:32:05.796198 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 17:32:05.796198 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 17:32:05.796198 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 17:32:05.797151 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 17:32:05.797151 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:32:05.797151 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:32:05.797151 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:32:05.797151 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:32:05.797151 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:32:05.797151 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:32:05.797151 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:32:05.797151 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:32:05.797151 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:32:05.797151 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:32:05.797151 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:32:05.797151 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:32:05.797151 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:32:05.797151 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:32:05.797151 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:32:05.797151 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:32:05.798078 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 17:32:05.798078 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 17:32:05.798078 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 17:32:05.798078 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 17:32:05.798078 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 17:32:05.798078 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 17:32:05.798078 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:32:05.798078 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:32:05.798969 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 447/873] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:32:05 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 17:32:05.802961 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 426/873] 10.222.0.89 () {56 vars in 1111 bytes} [Thu Jul 28 17:32:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:32:05.803966 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c74a632-2bb8-40be-a6ed-09ab0fa9177c None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:05.822455 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb6dcce-c873-4f58-b1a1-5c025fbfd506 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:05.862266 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:05.943289 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c74a632-2bb8-40be-a6ed-09ab0fa9177c None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:05.946856 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb6dcce-c873-4f58-b1a1-5c025fbfd506 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:05.984684 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f65b5f0-94fa-4c92-8c80-68de05d86f98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:05.985333 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f65b5f0-94fa-4c92-8c80-68de05d86f98 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:05.985613 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f65b5f0-94fa-4c92-8c80-68de05d86f98 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:05.985883 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f65b5f0-94fa-4c92-8c80-68de05d86f98 None admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:05.986627 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f65b5f0-94fa-4c92-8c80-68de05d86f98 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:32:05.986931 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f65b5f0-94fa-4c92-8c80-68de05d86f98 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:05.988349 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f65b5f0-94fa-4c92-8c80-68de05d86f98 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:05.994633 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d149abc6-6c34-4a9b-a2fb-414f6683ac0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:05.995276 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d149abc6-6c34-4a9b-a2fb-414f6683ac0d None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:05.996422 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d149abc6-6c34-4a9b-a2fb-414f6683ac0d None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:05.996706 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d149abc6-6c34-4a9b-a2fb-414f6683ac0d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:05.997360 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d149abc6-6c34-4a9b-a2fb-414f6683ac0d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:06.000332 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d149abc6-6c34-4a9b-a2fb-414f6683ac0d None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:06.002484 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 448/874] 10.222.0.89 () {56 vars in 1148 bytes} [Thu Jul 28 17:32:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:32:06.009465 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-d149abc6-6c34-4a9b-a2fb-414f6683ac0d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 17:32:06.009465 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:32:06.009465 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 17:32:06.009465 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 17:32:06.009465 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 17:32:06.009465 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 17:32:06.009465 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 17:32:06.009465 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:32:06.009465 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 17:32:06.009465 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:32:06.009465 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 17:32:06.009465 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:32:06.009465 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:32:06.009465 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:32:06.009465 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:32:06.009465 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:32:06.009465 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:32:06.010502 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:32:06.010502 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:32:06.010502 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:32:06.010502 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:32:06.010502 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:32:06.010502 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:32:06.010502 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 17:32:06.010502 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 17:32:06.010502 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 17:32:06.010502 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 17:32:06.010502 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:32:06.010502 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:32:06.010502 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:32:06.010502 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:32:06.010502 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:32:06.010502 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:32:06.011374 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:32:06.011374 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:32:06.011374 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:32:06.011374 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:32:06.011374 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:32:06.011374 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:32:06.011374 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:32:06.011374 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:32:06.011374 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:32:06.011374 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:32:06.011374 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 17:32:06.011374 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:32:06.011374 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 17:32:06.011374 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:32:06.011374 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 427/875] 10.222.0.89 () {56 vars in 1126 bytes} [Thu Jul 28 17:32:05 2022] GET /identity/v3/services/object-store => generated 93 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:32:06.013628 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f65b5f0-94fa-4c92-8c80-68de05d86f98 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:06.017118 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d149abc6-6c34-4a9b-a2fb-414f6683ac0d None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:06.041314 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:06.122720 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:06.129616 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d149abc6-6c34-4a9b-a2fb-414f6683ac0d None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:06.158694 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f65b5f0-94fa-4c92-8c80-68de05d86f98 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:06.170704 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca02d32e-ce48-4a9e-a18b-d15d772ecca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:06.171327 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca02d32e-ce48-4a9e-a18b-d15d772ecca6 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:06.171812 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca02d32e-ce48-4a9e-a18b-d15d772ecca6 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:06.172156 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca02d32e-ce48-4a9e-a18b-d15d772ecca6 None admin] PATH_INFO: `/v3/services` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:06.172924 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca02d32e-ce48-4a9e-a18b-d15d772ecca6 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:32:06.173334 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca02d32e-ce48-4a9e-a18b-d15d772ecca6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:06.174859 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca02d32e-ce48-4a9e-a18b-d15d772ecca6 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:06.186040 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 428/876] 10.222.0.89 () {56 vars in 1122 bytes} [Thu Jul 28 17:32:06 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:32:06.187187 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca02d32e-ce48-4a9e-a18b-d15d772ecca6 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:06.208489 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8c971e-cc78-4053-bf4a-70cd5972d9e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:06.209417 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8c971e-cc78-4053-bf4a-70cd5972d9e1 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:06.209967 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8c971e-cc78-4053-bf4a-70cd5972d9e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:06.210426 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8c971e-cc78-4053-bf4a-70cd5972d9e1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:06.211942 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8c971e-cc78-4053-bf4a-70cd5972d9e1 None admin] RBAC: Adding query filter params (role.id=0a9ea6e6d6f94955a5dc949b0746a3f1, scope.project.id=04434e8535a64044996316f1d17dc0bc, user.id=c44387d13a694ee6bb47a13906003330) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:32:06.212779 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8c971e-cc78-4053-bf4a-70cd5972d9e1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:06.215176 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8c971e-cc78-4053-bf4a-70cd5972d9e1 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:06.229220 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 449/877] 10.222.0.89 () {56 vars in 1374 bytes} [Thu Jul 28 17:32:06 2022] GET /identity/v3/role_assignments?user.id=c44387d13a694ee6bb47a13906003330&scope.project.id=04434e8535a64044996316f1d17dc0bc&role.id=0a9ea6e6d6f94955a5dc949b0746a3f1 => generated 608 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:32:06.308772 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca02d32e-ce48-4a9e-a18b-d15d772ecca6 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:06.346061 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c63eac7-5892-4a54-a18b-48fb4d580b64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:06.346747 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c63eac7-5892-4a54-a18b-48fb4d580b64 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:06.347031 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c63eac7-5892-4a54-a18b-48fb4d580b64 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:06.347304 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c63eac7-5892-4a54-a18b-48fb4d580b64 None admin] PATH_INFO: `/v3/services` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:06.347995 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c63eac7-5892-4a54-a18b-48fb4d580b64 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:32:06.348298 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c63eac7-5892-4a54-a18b-48fb4d580b64 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:06.349725 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c63eac7-5892-4a54-a18b-48fb4d580b64 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:06.355300 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 429/878] 10.222.0.89 () {56 vars in 1122 bytes} [Thu Jul 28 17:32:06 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:32:07.125452 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e16746f-c1ed-4237-86f3-1d663ccc4e89 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:07.125953 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e16746f-c1ed-4237-86f3-1d663ccc4e89 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:07.125953 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e16746f-c1ed-4237-86f3-1d663ccc4e89 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:07.127522 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 450/879] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:32:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:32:07.133019 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a73d17-5952-4b36-bbcf-cd19f10fe97f None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:07.133472 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a73d17-5952-4b36-bbcf-cd19f10fe97f None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:07.133652 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a73d17-5952-4b36-bbcf-cd19f10fe97f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:07.158211 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-81a73d17-5952-4b36-bbcf-cd19f10fe97f None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:32:07.159607 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-81a73d17-5952-4b36-bbcf-cd19f10fe97f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:32:07.180959 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:07.208974 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24533864-a3a0-4425-a9a1-c554012df797 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:07.209301 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24533864-a3a0-4425-a9a1-c554012df797 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:07.209562 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24533864-a3a0-4425-a9a1-c554012df797 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:07.211161 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 451/880] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:32:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:32:07.215805 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04e8527-52dd-4f02-8803-325a9223975e None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:07.216093 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04e8527-52dd-4f02-8803-325a9223975e None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:07.216371 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04e8527-52dd-4f02-8803-325a9223975e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:07.236916 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-e04e8527-52dd-4f02-8803-325a9223975e None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:32:07.238282 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-e04e8527-52dd-4f02-8803-325a9223975e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:32:07.243625 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 430/881] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:32:07 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:32:07.266746 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a73d17-5952-4b36-bbcf-cd19f10fe97f None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:07.315884 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 452/882] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:32:07 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 17:32:07.337737 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04e8527-52dd-4f02-8803-325a9223975e None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:07.353547 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:07.394533 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a73d17-5952-4b36-bbcf-cd19f10fe97f None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:07.434559 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e5ff3a-4ce4-44b8-95ed-e1c4ccc9b330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:07.436182 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e5ff3a-4ce4-44b8-95ed-e1c4ccc9b330 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:07.436867 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e5ff3a-4ce4-44b8-95ed-e1c4ccc9b330 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:07.437380 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e5ff3a-4ce4-44b8-95ed-e1c4ccc9b330 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:07.438214 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e5ff3a-4ce4-44b8-95ed-e1c4ccc9b330 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:07.439780 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e5ff3a-4ce4-44b8-95ed-e1c4ccc9b330 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:07.448601 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04e8527-52dd-4f02-8803-325a9223975e None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:07.449590 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-07e5ff3a-4ce4-44b8-95ed-e1c4ccc9b330 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 17:32:07.449590 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:32:07.449590 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 17:32:07.449590 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 17:32:07.449590 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 17:32:07.449590 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 17:32:07.449590 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 17:32:07.449590 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:32:07.449590 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 17:32:07.449590 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:32:07.449590 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 17:32:07.449590 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:32:07.449590 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:32:07.449590 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:32:07.449590 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:32:07.449590 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:32:07.449590 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:32:07.450599 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:32:07.450599 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:32:07.450599 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:32:07.450599 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:32:07.450599 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:32:07.450599 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:32:07.450599 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 17:32:07.450599 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 17:32:07.450599 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 17:32:07.450599 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 17:32:07.450599 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:32:07.450599 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:32:07.450599 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:32:07.450599 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:32:07.450599 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:32:07.450599 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:32:07.452047 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:32:07.452047 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:32:07.452047 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:32:07.452047 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:32:07.452047 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:32:07.452047 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:32:07.452047 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:32:07.452047 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:32:07.452047 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:32:07.452047 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:32:07.452047 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 17:32:07.452047 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:32:07.452047 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 17:32:07.452047 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:32:07.455158 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 431/883] 10.222.0.89 () {56 vars in 1114 bytes} [Thu Jul 28 17:32:07 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:32:07.462691 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e5ff3a-4ce4-44b8-95ed-e1c4ccc9b330 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:07.493775 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:07.497877 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffb4987-9673-4486-b1c1-b4bf2f9eab99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:07.498915 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffb4987-9673-4486-b1c1-b4bf2f9eab99 None admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:07.499781 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffb4987-9673-4486-b1c1-b4bf2f9eab99 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:07.500633 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffb4987-9673-4486-b1c1-b4bf2f9eab99 None admin] PATH_INFO: `/v3/services` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:07.501603 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffb4987-9673-4486-b1c1-b4bf2f9eab99 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:07.503499 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffb4987-9673-4486-b1c1-b4bf2f9eab99 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:07.522775 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 453/884] 10.222.0.89 () {60 vars in 1141 bytes} [Thu Jul 28 17:32:07 2022] POST /identity/v3/services => generated 269 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 17:32:07.599052 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e5ff3a-4ce4-44b8-95ed-e1c4ccc9b330 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:07.645382 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d36377-eeba-4d5b-b9d7-0955e754328f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:07.646307 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d36377-eeba-4d5b-b9d7-0955e754328f None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:07.646770 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d36377-eeba-4d5b-b9d7-0955e754328f None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:07.647268 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d36377-eeba-4d5b-b9d7-0955e754328f None admin] PATH_INFO: `/v3/services` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:07.648202 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d36377-eeba-4d5b-b9d7-0955e754328f None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:32:07.648707 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d36377-eeba-4d5b-b9d7-0955e754328f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:07.650299 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d36377-eeba-4d5b-b9d7-0955e754328f None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:07.657394 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 432/885] 10.222.0.89 () {56 vars in 1114 bytes} [Thu Jul 28 17:32:07 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:32:07.664924 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffb4987-9673-4486-b1c1-b4bf2f9eab99 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:07.740631 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:07.779337 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffb4987-9673-4486-b1c1-b4bf2f9eab99 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:07.821253 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b463dc97-57cb-4fc7-878c-0f2e016c6880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:07.821882 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b463dc97-57cb-4fc7-878c-0f2e016c6880 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:07.822163 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b463dc97-57cb-4fc7-878c-0f2e016c6880 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:07.822440 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b463dc97-57cb-4fc7-878c-0f2e016c6880 None admin] PATH_INFO: `/v3/services` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:07.823120 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b463dc97-57cb-4fc7-878c-0f2e016c6880 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:32:07.823427 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b463dc97-57cb-4fc7-878c-0f2e016c6880 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:07.827510 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b463dc97-57cb-4fc7-878c-0f2e016c6880 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:07.841296 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 454/886] 10.222.0.89 () {56 vars in 1114 bytes} [Thu Jul 28 17:32:07 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:32:07.844423 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d36377-eeba-4d5b-b9d7-0955e754328f None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:07.931465 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:07.953807 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d36377-eeba-4d5b-b9d7-0955e754328f None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:07.990525 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feafc3ff-b372-4b8f-8036-9f1ff45a7d61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:07.991547 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feafc3ff-b372-4b8f-8036-9f1ff45a7d61 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:07.992034 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feafc3ff-b372-4b8f-8036-9f1ff45a7d61 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:07.992566 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feafc3ff-b372-4b8f-8036-9f1ff45a7d61 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:07.993572 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feafc3ff-b372-4b8f-8036-9f1ff45a7d61 None admin] RBAC: Adding query filter params (interface=public, service_id=e0a5eb5b4a5e4c30a2e969736b8d3231, region_id=RegionOne) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:32:07.994103 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feafc3ff-b372-4b8f-8036-9f1ff45a7d61 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:07.995998 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feafc3ff-b372-4b8f-8036-9f1ff45a7d61 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:08.001911 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 433/887] 10.222.0.89 () {56 vars in 1251 bytes} [Thu Jul 28 17:32:07 2022] GET /identity/v3/endpoints?service_id=e0a5eb5b4a5e4c30a2e969736b8d3231&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:32:08.007891 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b463dc97-57cb-4fc7-878c-0f2e016c6880 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:08.115030 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b463dc97-57cb-4fc7-878c-0f2e016c6880 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:08.151063 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c6fdfc-99df-4ceb-b01d-e7ceabf3461c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:08.152493 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c6fdfc-99df-4ceb-b01d-e7ceabf3461c None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:08.153070 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c6fdfc-99df-4ceb-b01d-e7ceabf3461c None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:08.153569 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c6fdfc-99df-4ceb-b01d-e7ceabf3461c None admin] PATH_INFO: `/v3/services` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:08.154496 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c6fdfc-99df-4ceb-b01d-e7ceabf3461c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:08.156060 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c6fdfc-99df-4ceb-b01d-e7ceabf3461c None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:08.157382 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:08.168485 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 455/888] 10.222.0.89 () {56 vars in 1087 bytes} [Thu Jul 28 17:32:08 2022] GET /identity/v3/services => generated 1268 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:32:08.442881 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a01183-df11-4e4c-a1e8-46a6ebeec7f1 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:08.443401 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a01183-df11-4e4c-a1e8-46a6ebeec7f1 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:08.443482 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a01183-df11-4e4c-a1e8-46a6ebeec7f1 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:08.445023 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 434/889] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:32:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:32:08.450223 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b852c5c6-6c74-45bd-8bea-34497e7c8ac8 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:08.450516 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b852c5c6-6c74-45bd-8bea-34497e7c8ac8 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:08.450792 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b852c5c6-6c74-45bd-8bea-34497e7c8ac8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:08.469849 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-b852c5c6-6c74-45bd-8bea-34497e7c8ac8 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:32:08.471057 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-b852c5c6-6c74-45bd-8bea-34497e7c8ac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:32:08.479352 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03918ffd-e6d8-47d1-ab55-340805dafc70 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:08.479628 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03918ffd-e6d8-47d1-ab55-340805dafc70 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:08.479904 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03918ffd-e6d8-47d1-ab55-340805dafc70 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:08.481565 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 435/890] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:32:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:32:08.486968 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc32dec-1f4e-49d2-a358-af4e3c7ec004 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:08.487250 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc32dec-1f4e-49d2-a358-af4e3c7ec004 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:08.487640 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc32dec-1f4e-49d2-a358-af4e3c7ec004 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:08.510242 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-6bc32dec-1f4e-49d2-a358-af4e3c7ec004 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:32:08.512166 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-6bc32dec-1f4e-49d2-a358-af4e3c7ec004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:32:08.558078 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 456/891] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:32:08 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:32:08.581953 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b852c5c6-6c74-45bd-8bea-34497e7c8ac8 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:08.590786 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 436/892] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:32:08 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 17:32:08.613939 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc32dec-1f4e-49d2-a358-af4e3c7ec004 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:08.694855 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:08.703276 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b852c5c6-6c74-45bd-8bea-34497e7c8ac8 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:08.726179 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc32dec-1f4e-49d2-a358-af4e3c7ec004 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:08.741045 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:08.752794 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727a0fba-031e-42b7-ab3f-7728daad733e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:08.753882 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727a0fba-031e-42b7-ab3f-7728daad733e None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:08.754675 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727a0fba-031e-42b7-ab3f-7728daad733e None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:08.755112 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727a0fba-031e-42b7-ab3f-7728daad733e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:08.768617 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df3c31e-de0d-49f2-99b4-d1b20b8aa335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:08.769529 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df3c31e-de0d-49f2-99b4-d1b20b8aa335 None admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:08.769663 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727a0fba-031e-42b7-ab3f-7728daad733e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:32:08.770081 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df3c31e-de0d-49f2-99b4-d1b20b8aa335 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:08.770511 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df3c31e-de0d-49f2-99b4-d1b20b8aa335 None admin] PATH_INFO: `/v3/services` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:08.771362 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df3c31e-de0d-49f2-99b4-d1b20b8aa335 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:08.772888 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df3c31e-de0d-49f2-99b4-d1b20b8aa335 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:08.774540 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-727a0fba-031e-42b7-ab3f-7728daad733e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 17:32:08.774540 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:32:08.774540 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:32:08.774540 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:32:08.774540 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:32:08.774540 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:32:08.774540 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:32:08.774540 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:32:08.774540 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:32:08.774540 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:32:08.774540 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:32:08.774540 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:32:08.774540 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:32:08.774540 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:32:08.774540 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:32:08.774540 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:32:08.775759 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:32:08.775759 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:32:08.775759 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:32:08.775759 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:32:08.775759 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:32:08.775759 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:32:08.775759 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:32:08.775759 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:32:08.775759 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:32:08.775759 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:32:08.775759 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:32:08.775759 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( Jul 28 17:32:08.775759 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:32:08.775759 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:32:08.775759 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:32:08.775759 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:32:08.776874 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:32:08.776874 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:32:08.776874 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:32:08.776874 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:32:08.776874 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:32:08.776874 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:32:08.776874 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:32:08.776874 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:32:08.776874 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:32:08.776874 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:32:08.776874 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 17:32:08.776874 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:32:08.781640 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 457/893] 10.222.0.89 () {56 vars in 1120 bytes} [Thu Jul 28 17:32:08 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:32:08.783622 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727a0fba-031e-42b7-ab3f-7728daad733e None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:08.799957 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 437/894] 10.222.0.89 () {60 vars in 1141 bytes} [Thu Jul 28 17:32:08 2022] POST /identity/v3/services => generated 241 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:32:08.914156 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727a0fba-031e-42b7-ab3f-7728daad733e None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:08.960615 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fbc0bf-52b8-443f-9796-239081301a80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:08.961184 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fbc0bf-52b8-443f-9796-239081301a80 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:08.961475 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fbc0bf-52b8-443f-9796-239081301a80 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:08.961757 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fbc0bf-52b8-443f-9796-239081301a80 None admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:08.962471 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fbc0bf-52b8-443f-9796-239081301a80 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:32:08.962773 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fbc0bf-52b8-443f-9796-239081301a80 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:08.964158 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fbc0bf-52b8-443f-9796-239081301a80 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:08.974128 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 458/895] 10.222.0.89 () {56 vars in 1115 bytes} [Thu Jul 28 17:32:08 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:32:08.979660 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df3c31e-de0d-49f2-99b4-d1b20b8aa335 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:09.086872 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df3c31e-de0d-49f2-99b4-d1b20b8aa335 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:09.124212 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fbdaeb-2ed8-45e7-a534-13d976acfa0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:09.124706 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fbdaeb-2ed8-45e7-a534-13d976acfa0d None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:09.124977 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fbdaeb-2ed8-45e7-a534-13d976acfa0d None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:09.125274 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fbdaeb-2ed8-45e7-a534-13d976acfa0d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:09.127364 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:09.139199 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fbdaeb-2ed8-45e7-a534-13d976acfa0d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:09.140308 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fbdaeb-2ed8-45e7-a534-13d976acfa0d None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:09.156898 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-c8fbdaeb-2ed8-45e7-a534-13d976acfa0d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:32:09.156898 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:32:09.156898 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:32:09.156898 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:32:09.156898 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:32:09.156898 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:32:09.156898 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:32:09.156898 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:32:09.156898 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:32:09.156898 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:32:09.156898 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:32:09.156898 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:32:09.156898 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:32:09.156898 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:32:09.156898 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:32:09.157890 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:32:09.157890 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:32:09.157890 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:32:09.157890 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:32:09.157890 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:32:09.157890 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:32:09.157890 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:32:09.157890 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:32:09.157890 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:32:09.157890 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:32:09.157890 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:32:09.157890 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:32:09.157890 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:32:09.157890 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:32:09.157890 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:32:09.157890 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:32:09.157890 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:32:09.158783 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:32:09.158783 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:32:09.158783 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:32:09.158783 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:32:09.160962 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 438/896] 10.222.0.89 () {56 vars in 1108 bytes} [Thu Jul 28 17:32:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:32:09.166745 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fbc0bf-52b8-443f-9796-239081301a80 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:09.220672 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:09.276761 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fbc0bf-52b8-443f-9796-239081301a80 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:09.312950 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f3abeb-db27-4a91-9aba-08c97c6e3aef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:09.313601 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f3abeb-db27-4a91-9aba-08c97c6e3aef None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:09.313883 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f3abeb-db27-4a91-9aba-08c97c6e3aef None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:09.314155 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f3abeb-db27-4a91-9aba-08c97c6e3aef None admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:09.314847 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f3abeb-db27-4a91-9aba-08c97c6e3aef None admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:32:09.315149 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f3abeb-db27-4a91-9aba-08c97c6e3aef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:09.316516 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f3abeb-db27-4a91-9aba-08c97c6e3aef None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:09.330149 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 459/897] 10.222.0.89 () {56 vars in 1109 bytes} [Thu Jul 28 17:32:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:32:09.335615 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fbdaeb-2ed8-45e7-a534-13d976acfa0d None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:09.448937 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fbdaeb-2ed8-45e7-a534-13d976acfa0d None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:09.485187 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d40cb3-4912-4d94-9efb-b599e5ba001c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:09.485776 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d40cb3-4912-4d94-9efb-b599e5ba001c None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:09.486056 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d40cb3-4912-4d94-9efb-b599e5ba001c None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:09.486331 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d40cb3-4912-4d94-9efb-b599e5ba001c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:09.494402 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d40cb3-4912-4d94-9efb-b599e5ba001c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:09.496434 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d40cb3-4912-4d94-9efb-b599e5ba001c None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:09.509858 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-a7d40cb3-4912-4d94-9efb-b599e5ba001c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 17:32:09.509858 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:32:09.509858 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:32:09.509858 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:32:09.509858 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:32:09.509858 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:32:09.509858 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:32:09.509858 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:32:09.509858 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:32:09.509858 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:32:09.509858 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:32:09.509858 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:32:09.509858 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 17:32:09.509858 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 17:32:09.509858 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 17:32:09.509858 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 17:32:09.510922 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:32:09.510922 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:32:09.510922 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 17:32:09.510922 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:32:09.510922 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 17:32:09.510922 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:32:09.510922 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:32:09.510922 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:32:09.510922 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:32:09.510922 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:32:09.510922 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:32:09.510922 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:32:09.510922 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:32:09.510922 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:32:09.510922 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:32:09.510922 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:32:09.510922 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:32:09.510922 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:32:09.511909 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:32:09.511909 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:32:09.511909 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 17:32:09.511909 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 17:32:09.511909 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 17:32:09.511909 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 17:32:09.511909 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 17:32:09.511909 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 17:32:09.511909 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 17:32:09.511909 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:32:09.511909 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 439/898] 10.222.0.89 () {56 vars in 1117 bytes} [Thu Jul 28 17:32:09 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:32:09.518405 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f3abeb-db27-4a91-9aba-08c97c6e3aef None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:09.624455 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f3abeb-db27-4a91-9aba-08c97c6e3aef None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:09.632325 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:09.664187 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5171486e-10df-4d9e-93dc-492bc603de83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:09.665140 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5171486e-10df-4d9e-93dc-492bc603de83 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:09.665647 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5171486e-10df-4d9e-93dc-492bc603de83 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:09.666113 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5171486e-10df-4d9e-93dc-492bc603de83 None admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:09.667100 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5171486e-10df-4d9e-93dc-492bc603de83 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:32:09.667585 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5171486e-10df-4d9e-93dc-492bc603de83 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:09.669228 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5171486e-10df-4d9e-93dc-492bc603de83 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:09.689567 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 460/899] 10.222.0.89 () {56 vars in 1149 bytes} [Thu Jul 28 17:32:09 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:32:09.701543 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d40cb3-4912-4d94-9efb-b599e5ba001c None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:09.744979 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:09.777759 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe1a281-dc49-47f3-a218-688db99bdba1 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:09.778399 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe1a281-dc49-47f3-a218-688db99bdba1 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:09.778909 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe1a281-dc49-47f3-a218-688db99bdba1 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:09.785272 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 461/900] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:32:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:32:09.787217 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba2fb22-96e2-47f6-a66c-6a1795557040 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:09.787708 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba2fb22-96e2-47f6-a66c-6a1795557040 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:09.788187 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba2fb22-96e2-47f6-a66c-6a1795557040 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:09.802065 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d40cb3-4912-4d94-9efb-b599e5ba001c None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:09.814680 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-6ba2fb22-96e2-47f6-a66c-6a1795557040 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:32:09.816280 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-6ba2fb22-96e2-47f6-a66c-6a1795557040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:32:09.837177 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f34395-0398-4300-98ac-bbf7d0bd5d43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:09.837820 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f34395-0398-4300-98ac-bbf7d0bd5d43 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:09.838154 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f34395-0398-4300-98ac-bbf7d0bd5d43 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:09.838436 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f34395-0398-4300-98ac-bbf7d0bd5d43 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:09.906798 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 462/901] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:32:09 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 17:32:09.924187 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f34395-0398-4300-98ac-bbf7d0bd5d43 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:09.925670 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f34395-0398-4300-98ac-bbf7d0bd5d43 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:09.930364 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba2fb22-96e2-47f6-a66c-6a1795557040 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:09.984192 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:10.053687 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 440/902] 10.222.0.89 () {58 vars in 1282 bytes} [Thu Jul 28 17:32:09 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 17:32:10.062178 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f34395-0398-4300-98ac-bbf7d0bd5d43 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:10.065067 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:10.073105 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba2fb22-96e2-47f6-a66c-6a1795557040 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:10.112451 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b43e71e-7046-45ac-a2c8-a76320e2ea7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:10.113571 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b43e71e-7046-45ac-a2c8-a76320e2ea7b None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:10.114064 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b43e71e-7046-45ac-a2c8-a76320e2ea7b None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:10.114533 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b43e71e-7046-45ac-a2c8-a76320e2ea7b None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:10.115394 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b43e71e-7046-45ac-a2c8-a76320e2ea7b None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:10.116961 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b43e71e-7046-45ac-a2c8-a76320e2ea7b None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:10.127715 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-1b43e71e-7046-45ac-a2c8-a76320e2ea7b None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 17:32:10.127715 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:32:10.127715 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 17:32:10.127715 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 17:32:10.127715 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 17:32:10.127715 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 17:32:10.127715 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 17:32:10.127715 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:32:10.127715 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 17:32:10.127715 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:32:10.127715 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 17:32:10.127715 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:32:10.127715 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:32:10.127715 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:32:10.127715 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:32:10.127715 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:32:10.128772 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:32:10.128772 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:32:10.128772 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:32:10.128772 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:32:10.128772 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:32:10.128772 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:32:10.128772 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:32:10.128772 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 17:32:10.128772 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 17:32:10.128772 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 17:32:10.128772 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 17:32:10.128772 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:32:10.128772 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:32:10.128772 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:32:10.128772 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:32:10.128772 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:32:10.129678 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:32:10.129678 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:32:10.129678 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( Jul 28 17:32:10.129678 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:32:10.129678 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:32:10.129678 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:32:10.129678 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:32:10.129678 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:32:10.129678 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:32:10.129678 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:32:10.129678 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:32:10.129678 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 17:32:10.129678 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:32:10.129678 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 17:32:10.129678 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:32:10.132200 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 463/903] 10.222.0.89 () {56 vars in 1132 bytes} [Thu Jul 28 17:32:09 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:32:10.140258 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b43e71e-7046-45ac-a2c8-a76320e2ea7b None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:10.178746 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f34395-0398-4300-98ac-bbf7d0bd5d43 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:10.216331 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfbdb48-184e-4e4b-8863-13cc01c7cebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:10.217602 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfbdb48-184e-4e4b-8863-13cc01c7cebf None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:10.217602 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfbdb48-184e-4e4b-8863-13cc01c7cebf None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:10.218183 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfbdb48-184e-4e4b-8863-13cc01c7cebf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:10.228757 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfbdb48-184e-4e4b-8863-13cc01c7cebf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:10.230353 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfbdb48-184e-4e4b-8863-13cc01c7cebf None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:10.239086 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:10.248041 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-7cfbdb48-184e-4e4b-8863-13cc01c7cebf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:32:10.248041 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:32:10.248041 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:32:10.248041 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:32:10.248041 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:32:10.248041 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:32:10.248041 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:32:10.248041 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:32:10.248041 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:32:10.248041 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:32:10.248041 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:32:10.248041 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:32:10.248041 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:32:10.248041 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:32:10.248041 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:32:10.249723 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:32:10.249723 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:32:10.249723 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:32:10.249723 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:32:10.249723 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:32:10.249723 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:32:10.249723 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:32:10.249723 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:32:10.249723 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:32:10.249723 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:32:10.249723 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:32:10.249723 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:32:10.249723 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:32:10.249723 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:32:10.249723 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:32:10.249723 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:32:10.249723 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:32:10.250771 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:32:10.250771 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:32:10.250771 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:32:10.250771 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:32:10.254623 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 441/904] 10.222.0.89 () {56 vars in 1108 bytes} [Thu Jul 28 17:32:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:32:10.255875 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfbdb48-184e-4e4b-8863-13cc01c7cebf None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:10.262756 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b43e71e-7046-45ac-a2c8-a76320e2ea7b None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:10.301542 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04945dfd-88be-447c-b234-ad465d691a73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:10.302426 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04945dfd-88be-447c-b234-ad465d691a73 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:10.302895 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04945dfd-88be-447c-b234-ad465d691a73 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:10.303414 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04945dfd-88be-447c-b234-ad465d691a73 None admin] PATH_INFO: `/v3/services` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:10.304287 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04945dfd-88be-447c-b234-ad465d691a73 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:32:10.304734 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04945dfd-88be-447c-b234-ad465d691a73 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:10.306334 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04945dfd-88be-447c-b234-ad465d691a73 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:10.317021 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 464/905] 10.222.0.89 () {56 vars in 1126 bytes} [Thu Jul 28 17:32:10 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:32:10.318115 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04945dfd-88be-447c-b234-ad465d691a73 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:10.353830 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:10.377820 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfbdb48-184e-4e4b-8863-13cc01c7cebf None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:10.414858 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd929ed-318f-46aa-87fd-c10e14f36256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:10.415473 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd929ed-318f-46aa-87fd-c10e14f36256 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:10.415747 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd929ed-318f-46aa-87fd-c10e14f36256 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:10.416016 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd929ed-318f-46aa-87fd-c10e14f36256 None admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:10.416992 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd929ed-318f-46aa-87fd-c10e14f36256 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:32:10.417392 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd929ed-318f-46aa-87fd-c10e14f36256 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:10.418770 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd929ed-318f-46aa-87fd-c10e14f36256 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:10.431611 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 442/906] 10.222.0.89 () {56 vars in 1109 bytes} [Thu Jul 28 17:32:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:32:10.433232 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fff73e-be1f-419c-8f47-d960a4c13b00 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:10.433708 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fff73e-be1f-419c-8f47-d960a4c13b00 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:10.434176 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fff73e-be1f-419c-8f47-d960a4c13b00 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:10.436985 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 443/907] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:32:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:32:10.437992 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fff73e-be1f-419c-8f47-d960a4c13b00 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:10.452055 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04945dfd-88be-447c-b234-ad465d691a73 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:10.487327 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664f5d06-9eef-4453-967a-2069c1b87071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:10.487952 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664f5d06-9eef-4453-967a-2069c1b87071 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:10.488233 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664f5d06-9eef-4453-967a-2069c1b87071 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:10.488514 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664f5d06-9eef-4453-967a-2069c1b87071 None admin] PATH_INFO: `/v3/services` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:10.489364 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664f5d06-9eef-4453-967a-2069c1b87071 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:32:10.489530 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664f5d06-9eef-4453-967a-2069c1b87071 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:10.490911 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664f5d06-9eef-4453-967a-2069c1b87071 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:10.503082 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 465/908] 10.222.0.89 () {56 vars in 1126 bytes} [Thu Jul 28 17:32:10 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:32:10.504472 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9a1da0-a320-4444-aed3-e22afa20f905 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:10.504769 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9a1da0-a320-4444-aed3-e22afa20f905 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:10.505042 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9a1da0-a320-4444-aed3-e22afa20f905 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:10.506969 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:10.526435 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-cf9a1da0-a320-4444-aed3-e22afa20f905 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:32:10.527643 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-cf9a1da0-a320-4444-aed3-e22afa20f905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:32:10.555109 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fff73e-be1f-419c-8f47-d960a4c13b00 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:10.593904 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901ef3a5-037e-4324-9ccc-ec81050275a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:10.594550 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901ef3a5-037e-4324-9ccc-ec81050275a4 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:10.594826 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901ef3a5-037e-4324-9ccc-ec81050275a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:10.595094 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901ef3a5-037e-4324-9ccc-ec81050275a4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:10.601789 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 466/909] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:32:10 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:32:10.601940 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901ef3a5-037e-4324-9ccc-ec81050275a4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:10.603055 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9a1da0-a320-4444-aed3-e22afa20f905 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:10.603289 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901ef3a5-037e-4324-9ccc-ec81050275a4 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:10.621962 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:10.624008 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-901ef3a5-037e-4324-9ccc-ec81050275a4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:32:10.624008 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:32:10.624008 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:32:10.624008 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:32:10.624008 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:32:10.624008 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:32:10.624008 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:32:10.624008 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:32:10.624008 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:32:10.624008 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:32:10.624008 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:32:10.624008 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:32:10.624008 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 17:32:10.624008 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 17:32:10.624008 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 17:32:10.624978 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 17:32:10.624978 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:32:10.624978 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:32:10.624978 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:32:10.624978 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:32:10.624978 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:32:10.624978 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:32:10.624978 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:32:10.624978 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:32:10.624978 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:32:10.624978 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:32:10.624978 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:32:10.624978 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:32:10.624978 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:32:10.624978 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:32:10.624978 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:32:10.624978 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:32:10.625967 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 17:32:10.625967 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 17:32:10.625967 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 17:32:10.625967 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 17:32:10.625967 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 17:32:10.625967 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 17:32:10.625967 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:32:10.625967 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:32:10.630608 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 444/910] 10.222.0.89 () {56 vars in 1111 bytes} [Thu Jul 28 17:32:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:32:10.631661 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901ef3a5-037e-4324-9ccc-ec81050275a4 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:10.721073 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9a1da0-a320-4444-aed3-e22afa20f905 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:10.744789 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901ef3a5-037e-4324-9ccc-ec81050275a4 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:10.757792 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4ebcfa-6eec-49c2-8fdb-17dc794dc918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:10.758753 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4ebcfa-6eec-49c2-8fdb-17dc794dc918 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:10.759277 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4ebcfa-6eec-49c2-8fdb-17dc794dc918 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:10.759621 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4ebcfa-6eec-49c2-8fdb-17dc794dc918 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:10.760457 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4ebcfa-6eec-49c2-8fdb-17dc794dc918 None admin] RBAC: Adding query filter params (interface=public, service_id=1e45365578284f459092ce7d2b9a02e6, region_id=RegionOne) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:32:10.760831 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4ebcfa-6eec-49c2-8fdb-17dc794dc918 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:10.762169 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4ebcfa-6eec-49c2-8fdb-17dc794dc918 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:10.763886 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:10.771977 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 467/911] 10.222.0.89 () {56 vars in 1251 bytes} [Thu Jul 28 17:32:10 2022] GET /identity/v3/endpoints?service_id=1e45365578284f459092ce7d2b9a02e6&interface=public®ion_id=RegionOne => generated 194 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:32:10.771977 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4ebcfa-6eec-49c2-8fdb-17dc794dc918 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:10.788705 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3232b8-907c-4a9c-b2b3-28929fc36dc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:10.789353 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3232b8-907c-4a9c-b2b3-28929fc36dc3 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:10.789634 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3232b8-907c-4a9c-b2b3-28929fc36dc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:10.789905 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3232b8-907c-4a9c-b2b3-28929fc36dc3 None admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:10.790662 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3232b8-907c-4a9c-b2b3-28929fc36dc3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:32:10.790955 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3232b8-907c-4a9c-b2b3-28929fc36dc3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:10.792381 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3232b8-907c-4a9c-b2b3-28929fc36dc3 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:10.805579 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 445/912] 10.222.0.89 () {56 vars in 1148 bytes} [Thu Jul 28 17:32:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:32:10.806308 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3232b8-907c-4a9c-b2b3-28929fc36dc3 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:10.874918 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:10.895877 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4ebcfa-6eec-49c2-8fdb-17dc794dc918 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:10.914432 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3232b8-907c-4a9c-b2b3-28929fc36dc3 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:10.930360 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e19e46-bae0-47bb-b84a-575960ee034a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:10.931009 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e19e46-bae0-47bb-b84a-575960ee034a None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:10.931289 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e19e46-bae0-47bb-b84a-575960ee034a None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:10.931565 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e19e46-bae0-47bb-b84a-575960ee034a None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:10.932193 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e19e46-bae0-47bb-b84a-575960ee034a None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:10.933604 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e19e46-bae0-47bb-b84a-575960ee034a None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:10.943470 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-e4e19e46-bae0-47bb-b84a-575960ee034a None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 17:32:10.943470 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:32:10.943470 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 17:32:10.943470 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 17:32:10.943470 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 17:32:10.943470 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 17:32:10.943470 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 17:32:10.943470 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:32:10.943470 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 17:32:10.943470 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:32:10.943470 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 17:32:10.943470 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:32:10.943470 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:32:10.943470 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:32:10.943470 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:32:10.943470 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:32:10.943470 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:32:10.944733 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:32:10.944733 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:32:10.944733 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:32:10.944733 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:32:10.944733 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:32:10.944733 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:32:10.944733 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 17:32:10.944733 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 17:32:10.944733 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 17:32:10.944733 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 17:32:10.944733 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:32:10.944733 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:32:10.944733 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:32:10.944733 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:32:10.944733 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:32:10.944733 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:32:10.945712 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:32:10.945712 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( Jul 28 17:32:10.945712 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:32:10.945712 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:32:10.945712 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:32:10.945712 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:32:10.945712 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:32:10.945712 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:32:10.945712 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:32:10.945712 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:32:10.945712 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 17:32:10.945712 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:32:10.945712 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 17:32:10.945712 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:32:10.949491 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 468/913] 10.222.0.89 () {56 vars in 1114 bytes} [Thu Jul 28 17:32:10 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:32:10.950731 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e19e46-bae0-47bb-b84a-575960ee034a None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:10.955971 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3624f2f-b9d5-4050-945c-4eabfcee1ff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:10.956843 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3624f2f-b9d5-4050-945c-4eabfcee1ff7 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:10.957342 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3624f2f-b9d5-4050-945c-4eabfcee1ff7 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:10.957800 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3624f2f-b9d5-4050-945c-4eabfcee1ff7 None admin] PATH_INFO: `/v3/services` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:10.958710 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3624f2f-b9d5-4050-945c-4eabfcee1ff7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:10.960453 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3624f2f-b9d5-4050-945c-4eabfcee1ff7 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:10.967561 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 446/914] 10.222.0.89 () {56 vars in 1087 bytes} [Thu Jul 28 17:32:10 2022] GET /identity/v3/services => generated 1497 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:32:10.968602 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3624f2f-b9d5-4050-945c-4eabfcee1ff7 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:11.039539 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:11.088363 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e19e46-bae0-47bb-b84a-575960ee034a None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:11.113603 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3624f2f-b9d5-4050-945c-4eabfcee1ff7 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:11.130227 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff959c9-d5fd-4392-8ba8-7c407ea99c42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:11.130845 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff959c9-d5fd-4392-8ba8-7c407ea99c42 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:11.131122 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff959c9-d5fd-4392-8ba8-7c407ea99c42 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:11.131972 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff959c9-d5fd-4392-8ba8-7c407ea99c42 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:11.132196 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff959c9-d5fd-4392-8ba8-7c407ea99c42 None admin] RBAC: Adding query filter params (role.id=af4b58a9c88d4cb4861277524ab3e1ed, scope.project.id=04434e8535a64044996316f1d17dc0bc, user.id=c44387d13a694ee6bb47a13906003330) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:32:11.132510 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff959c9-d5fd-4392-8ba8-7c407ea99c42 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:11.133996 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff959c9-d5fd-4392-8ba8-7c407ea99c42 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:11.140650 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 469/915] 10.222.0.89 () {56 vars in 1374 bytes} [Thu Jul 28 17:32:10 2022] GET /identity/v3/role_assignments?user.id=c44387d13a694ee6bb47a13906003330&scope.project.id=04434e8535a64044996316f1d17dc0bc&role.id=af4b58a9c88d4cb4861277524ab3e1ed => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:32:11.141917 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcd4a01-a455-4435-90d4-cae28e069e69 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:11.142205 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcd4a01-a455-4435-90d4-cae28e069e69 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:11.142484 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcd4a01-a455-4435-90d4-cae28e069e69 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:11.143730 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 470/916] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:32:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:32:11.149980 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11d9743-0511-437d-9dc8-9009e6228415 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:11.149980 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11d9743-0511-437d-9dc8-9009e6228415 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:11.149980 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11d9743-0511-437d-9dc8-9009e6228415 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:11.161293 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bd2e51-72ed-4ee6-aab9-5e6ad6b79679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:11.162330 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bd2e51-72ed-4ee6-aab9-5e6ad6b79679 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:11.162330 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bd2e51-72ed-4ee6-aab9-5e6ad6b79679 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:11.162501 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bd2e51-72ed-4ee6-aab9-5e6ad6b79679 None admin] PATH_INFO: `/v3/services` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:11.163144 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bd2e51-72ed-4ee6-aab9-5e6ad6b79679 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:32:11.163480 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bd2e51-72ed-4ee6-aab9-5e6ad6b79679 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:11.164888 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bd2e51-72ed-4ee6-aab9-5e6ad6b79679 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:11.170544 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-a11d9743-0511-437d-9dc8-9009e6228415 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:32:11.171317 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 447/917] 10.222.0.89 () {56 vars in 1114 bytes} [Thu Jul 28 17:32:10 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:32:11.171804 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-a11d9743-0511-437d-9dc8-9009e6228415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:32:11.174312 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:11.178121 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bd2e51-72ed-4ee6-aab9-5e6ad6b79679 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:11.279644 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 471/918] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:32:11 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 17:32:11.305677 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bd2e51-72ed-4ee6-aab9-5e6ad6b79679 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:11.307889 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11d9743-0511-437d-9dc8-9009e6228415 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:11.318947 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:11.362121 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027d59ed-f273-4a9e-bd64-cfc35ba07e3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:11.362863 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027d59ed-f273-4a9e-bd64-cfc35ba07e3e None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:11.363150 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027d59ed-f273-4a9e-bd64-cfc35ba07e3e None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:11.363561 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027d59ed-f273-4a9e-bd64-cfc35ba07e3e None admin] PATH_INFO: `/v3/services` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:11.364357 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027d59ed-f273-4a9e-bd64-cfc35ba07e3e None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:32:11.364740 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027d59ed-f273-4a9e-bd64-cfc35ba07e3e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:11.366383 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027d59ed-f273-4a9e-bd64-cfc35ba07e3e None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:11.376473 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 448/919] 10.222.0.89 () {56 vars in 1114 bytes} [Thu Jul 28 17:32:11 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:32:11.382546 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027d59ed-f273-4a9e-bd64-cfc35ba07e3e None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:11.440909 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11d9743-0511-437d-9dc8-9009e6228415 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:11.487312 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d91748-f84c-40a5-a4a2-eee90853ca62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:11.488351 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d91748-f84c-40a5-a4a2-eee90853ca62 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:11.488839 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d91748-f84c-40a5-a4a2-eee90853ca62 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:11.489358 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d91748-f84c-40a5-a4a2-eee90853ca62 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:11.490310 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d91748-f84c-40a5-a4a2-eee90853ca62 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:11.491948 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d91748-f84c-40a5-a4a2-eee90853ca62 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:11.503585 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-c2d91748-f84c-40a5-a4a2-eee90853ca62 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 17:32:11.503585 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:32:11.503585 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 17:32:11.503585 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 17:32:11.503585 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 17:32:11.503585 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 17:32:11.503585 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 17:32:11.503585 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:32:11.503585 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 17:32:11.503585 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:32:11.503585 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 17:32:11.503585 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:32:11.503585 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:32:11.503585 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:32:11.503585 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:32:11.503585 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:32:11.503585 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:32:11.504651 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:32:11.504651 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:32:11.504651 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:32:11.504651 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:32:11.504651 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:32:11.504651 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:32:11.504651 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 17:32:11.504651 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 17:32:11.504651 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 17:32:11.504651 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 17:32:11.504651 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:32:11.504651 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:32:11.504651 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:32:11.504651 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:32:11.504651 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:32:11.504651 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:32:11.505983 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:32:11.505983 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( Jul 28 17:32:11.505983 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:32:11.505983 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:32:11.505983 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:32:11.505983 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:32:11.505983 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:32:11.505983 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:32:11.505983 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:32:11.505983 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:32:11.505983 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 17:32:11.505983 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:32:11.505983 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 17:32:11.505983 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:32:11.513239 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 472/920] 10.222.0.89 () {56 vars in 1126 bytes} [Thu Jul 28 17:32:11 2022] GET /identity/v3/services/object-store => generated 93 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:32:11.516355 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d91748-f84c-40a5-a4a2-eee90853ca62 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:11.519882 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:11.523835 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027d59ed-f273-4a9e-bd64-cfc35ba07e3e None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:11.570667 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf617dd9-6730-46d5-95d4-7d4d3b8edcc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:11.571294 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf617dd9-6730-46d5-95d4-7d4d3b8edcc9 None admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:11.571575 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf617dd9-6730-46d5-95d4-7d4d3b8edcc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:11.571854 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf617dd9-6730-46d5-95d4-7d4d3b8edcc9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:11.572836 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf617dd9-6730-46d5-95d4-7d4d3b8edcc9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:11.574946 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf617dd9-6730-46d5-95d4-7d4d3b8edcc9 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:11.603831 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 449/921] 10.222.0.89 () {60 vars in 1144 bytes} [Thu Jul 28 17:32:11 2022] POST /identity/v3/endpoints => generated 346 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 17:32:11.642527 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d91748-f84c-40a5-a4a2-eee90853ca62 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:11.692398 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f29a69-d11a-4952-beb0-79530a0dcab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:11.693149 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f29a69-d11a-4952-beb0-79530a0dcab7 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:11.693545 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f29a69-d11a-4952-beb0-79530a0dcab7 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:11.693835 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f29a69-d11a-4952-beb0-79530a0dcab7 None admin] PATH_INFO: `/v3/services` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:11.694703 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f29a69-d11a-4952-beb0-79530a0dcab7 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:32:11.695020 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f29a69-d11a-4952-beb0-79530a0dcab7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:11.696546 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f29a69-d11a-4952-beb0-79530a0dcab7 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:11.703921 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 473/922] 10.222.0.89 () {56 vars in 1122 bytes} [Thu Jul 28 17:32:11 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:32:11.709941 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf617dd9-6730-46d5-95d4-7d4d3b8edcc9 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:11.750307 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:11.857112 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf617dd9-6730-46d5-95d4-7d4d3b8edcc9 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:11.896870 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9b6799-c7d2-4331-a8b1-53491c95bc41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:11.897446 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9b6799-c7d2-4331-a8b1-53491c95bc41 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:11.897721 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9b6799-c7d2-4331-a8b1-53491c95bc41 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:11.897994 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9b6799-c7d2-4331-a8b1-53491c95bc41 None admin] PATH_INFO: `/v3/services` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:11.898674 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9b6799-c7d2-4331-a8b1-53491c95bc41 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:32:11.898971 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9b6799-c7d2-4331-a8b1-53491c95bc41 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:11.900395 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9b6799-c7d2-4331-a8b1-53491c95bc41 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:11.907474 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 450/923] 10.222.0.89 () {56 vars in 1122 bytes} [Thu Jul 28 17:32:11 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:32:11.913076 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f29a69-d11a-4952-beb0-79530a0dcab7 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:11.980979 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:12.024810 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f29a69-d11a-4952-beb0-79530a0dcab7 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:12.067666 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328b14a9-a73b-4975-8b78-e6ae024909a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:12.069431 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328b14a9-a73b-4975-8b78-e6ae024909a5 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:12.070268 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328b14a9-a73b-4975-8b78-e6ae024909a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:12.071095 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328b14a9-a73b-4975-8b78-e6ae024909a5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:12.072810 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328b14a9-a73b-4975-8b78-e6ae024909a5 None admin] RBAC: Adding query filter params (interface=public, service_id=1edcf516a5e14b8ca7e79aa7abe3f823, region_id=RegionOne) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:32:12.073727 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328b14a9-a73b-4975-8b78-e6ae024909a5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:12.077020 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328b14a9-a73b-4975-8b78-e6ae024909a5 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:12.083915 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 474/924] 10.222.0.89 () {56 vars in 1251 bytes} [Thu Jul 28 17:32:11 2022] GET /identity/v3/endpoints?service_id=1edcf516a5e14b8ca7e79aa7abe3f823&interface=public®ion_id=RegionOne => generated 194 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:32:12.089405 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9b6799-c7d2-4331-a8b1-53491c95bc41 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:12.204835 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:12.215941 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9b6799-c7d2-4331-a8b1-53491c95bc41 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:12.255140 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce14380-16dd-40d5-a914-71ddd99853de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:12.255756 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce14380-16dd-40d5-a914-71ddd99853de None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:12.256041 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce14380-16dd-40d5-a914-71ddd99853de None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:12.256313 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce14380-16dd-40d5-a914-71ddd99853de None admin] PATH_INFO: `/v3/services` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:12.256984 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce14380-16dd-40d5-a914-71ddd99853de None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:12.258411 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce14380-16dd-40d5-a914-71ddd99853de None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:12.265620 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 451/925] 10.222.0.89 () {56 vars in 1087 bytes} [Thu Jul 28 17:32:12 2022] GET /identity/v3/services => generated 1497 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:32:13.132788 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37c1a2f-acdb-4eb8-ad8c-71bc3e9a7d79 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:13.133311 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37c1a2f-acdb-4eb8-ad8c-71bc3e9a7d79 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:13.133311 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37c1a2f-acdb-4eb8-ad8c-71bc3e9a7d79 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:13.134962 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 475/926] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:32:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:32:13.141432 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd23a26-e253-45b1-8d32-fde4d98f39b8 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:13.141919 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd23a26-e253-45b1-8d32-fde4d98f39b8 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:13.142019 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd23a26-e253-45b1-8d32-fde4d98f39b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:13.160855 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-3dd23a26-e253-45b1-8d32-fde4d98f39b8 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:32:13.162104 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-3dd23a26-e253-45b1-8d32-fde4d98f39b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:32:13.234609 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 452/927] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:32:13 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:32:13.260310 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37c1a2f-acdb-4eb8-ad8c-71bc3e9a7d79 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:13.377204 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37c1a2f-acdb-4eb8-ad8c-71bc3e9a7d79 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:13.416947 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:13.418097 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820a5d71-71ed-45c7-8702-dbebf404426f None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:13.418097 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820a5d71-71ed-45c7-8702-dbebf404426f None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:13.418097 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820a5d71-71ed-45c7-8702-dbebf404426f None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:13.421283 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 453/928] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:32:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 17:32:13.427006 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbbcb63-4064-42ae-b609-77be2e076518 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:13.427250 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbbcb63-4064-42ae-b609-77be2e076518 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:13.427719 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbbcb63-4064-42ae-b609-77be2e076518 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:13.432213 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f720eca2-d9ae-4636-ba8e-def3d0a0a406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:13.433287 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f720eca2-d9ae-4636-ba8e-def3d0a0a406 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:13.433881 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f720eca2-d9ae-4636-ba8e-def3d0a0a406 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:13.434728 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f720eca2-d9ae-4636-ba8e-def3d0a0a406 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:13.435782 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f720eca2-d9ae-4636-ba8e-def3d0a0a406 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:13.437677 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f720eca2-d9ae-4636-ba8e-def3d0a0a406 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:13.451644 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-1bbbcb63-4064-42ae-b609-77be2e076518 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:32:13.453605 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-1bbbcb63-4064-42ae-b609-77be2e076518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:32:13.457854 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-f720eca2-d9ae-4636-ba8e-def3d0a0a406 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 17:32:13.457854 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:32:13.457854 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 17:32:13.457854 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 17:32:13.457854 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 17:32:13.457854 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 17:32:13.457854 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 17:32:13.457854 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:32:13.457854 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 17:32:13.457854 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:32:13.457854 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 17:32:13.457854 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:32:13.457854 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:32:13.457854 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:32:13.457854 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:32:13.457854 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:32:13.459698 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:32:13.459698 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:32:13.459698 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:32:13.459698 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:32:13.459698 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:32:13.459698 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:32:13.459698 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:32:13.459698 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 17:32:13.459698 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 17:32:13.459698 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 17:32:13.459698 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 17:32:13.459698 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:32:13.459698 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:32:13.459698 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:32:13.459698 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:32:13.459698 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:32:13.468114 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:32:13.468114 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:32:13.468114 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( Jul 28 17:32:13.468114 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:32:13.468114 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:32:13.468114 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:32:13.468114 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:32:13.468114 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:32:13.468114 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:32:13.468114 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:32:13.468114 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:32:13.468114 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 17:32:13.468114 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:32:13.468114 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 17:32:13.468114 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:32:13.468114 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 476/929] 10.222.0.89 () {56 vars in 1132 bytes} [Thu Jul 28 17:32:13 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:32:13.474286 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f720eca2-d9ae-4636-ba8e-def3d0a0a406 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:13.557249 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 454/930] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:32:13 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 17:32:13.574502 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbbcb63-4064-42ae-b609-77be2e076518 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:13.590681 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:13.617640 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f720eca2-d9ae-4636-ba8e-def3d0a0a406 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:13.658514 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d544bf1-02da-44ee-9ac8-11b74bc802fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:13.659493 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d544bf1-02da-44ee-9ac8-11b74bc802fc None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:13.660013 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d544bf1-02da-44ee-9ac8-11b74bc802fc None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:13.660499 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d544bf1-02da-44ee-9ac8-11b74bc802fc None admin] PATH_INFO: `/v3/services` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:13.661603 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d544bf1-02da-44ee-9ac8-11b74bc802fc None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:32:13.662118 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d544bf1-02da-44ee-9ac8-11b74bc802fc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:13.663675 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d544bf1-02da-44ee-9ac8-11b74bc802fc None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:13.669451 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 477/931] 10.222.0.89 () {56 vars in 1126 bytes} [Thu Jul 28 17:32:13 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:32:13.674887 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d544bf1-02da-44ee-9ac8-11b74bc802fc None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:13.684922 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbbcb63-4064-42ae-b609-77be2e076518 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:13.720912 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c15e197-14b5-4ac4-9191-073b34449487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:13.721591 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c15e197-14b5-4ac4-9191-073b34449487 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:13.721873 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c15e197-14b5-4ac4-9191-073b34449487 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:13.722147 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c15e197-14b5-4ac4-9191-073b34449487 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:13.733092 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c15e197-14b5-4ac4-9191-073b34449487 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:32:13.738387 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-8c15e197-14b5-4ac4-9191-073b34449487 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 17:32:13.738387 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:32:13.738387 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:32:13.738387 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:32:13.738387 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:32:13.738387 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:32:13.738387 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:32:13.738387 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:32:13.738387 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:32:13.738387 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:32:13.738387 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:32:13.738387 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:32:13.738387 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:32:13.738387 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:32:13.738387 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:32:13.738387 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:32:13.739344 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:32:13.739344 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:32:13.739344 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:32:13.739344 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:32:13.739344 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:32:13.739344 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:32:13.739344 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:32:13.739344 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:32:13.739344 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:32:13.739344 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:32:13.739344 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:32:13.739344 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:32:13.739344 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:32:13.739344 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:32:13.739344 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:32:13.739344 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:32:13.740186 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:32:13.740186 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:32:13.740186 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:32:13.740186 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:32:13.740186 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:32:13.740186 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:32:13.740186 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:32:13.740186 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:32:13.740186 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:32:13.740186 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:32:13.740186 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 17:32:13.740186 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:32:13.744744 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 455/932] 10.222.0.89 () {56 vars in 1120 bytes} [Thu Jul 28 17:32:13 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:32:13.745847 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:13.745965 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c15e197-14b5-4ac4-9191-073b34449487 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:13.801815 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d544bf1-02da-44ee-9ac8-11b74bc802fc None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:13.836565 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:13.857856 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef53cbc-9356-4316-a043-a2a65525d1c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:13.859280 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef53cbc-9356-4316-a043-a2a65525d1c4 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:13.859778 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef53cbc-9356-4316-a043-a2a65525d1c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:13.860224 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef53cbc-9356-4316-a043-a2a65525d1c4 None admin] PATH_INFO: `/v3/services` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:13.861068 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef53cbc-9356-4316-a043-a2a65525d1c4 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:32:13.861577 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef53cbc-9356-4316-a043-a2a65525d1c4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:13.863128 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef53cbc-9356-4316-a043-a2a65525d1c4 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:13.868483 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c15e197-14b5-4ac4-9191-073b34449487 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:13.868943 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 478/933] 10.222.0.89 () {56 vars in 1126 bytes} [Thu Jul 28 17:32:13 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:32:13.875043 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef53cbc-9356-4316-a043-a2a65525d1c4 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:13.914750 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb76256-eb49-447c-b741-bd36dde290b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:13.915619 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb76256-eb49-447c-b741-bd36dde290b8 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:13.915905 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb76256-eb49-447c-b741-bd36dde290b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:13.916204 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb76256-eb49-447c-b741-bd36dde290b8 None admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:13.916939 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb76256-eb49-447c-b741-bd36dde290b8 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:32:13.917288 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb76256-eb49-447c-b741-bd36dde290b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:13.918668 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb76256-eb49-447c-b741-bd36dde290b8 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:13.932234 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 456/934] 10.222.0.89 () {56 vars in 1115 bytes} [Thu Jul 28 17:32:13 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:32:13.932234 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51ecbca-4e44-4850-b720-ee7ef04fdcaa None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:13.932234 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51ecbca-4e44-4850-b720-ee7ef04fdcaa None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:13.932234 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51ecbca-4e44-4850-b720-ee7ef04fdcaa None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:13.932234 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 457/935] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:32:13 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:32:13.935358 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51ecbca-4e44-4850-b720-ee7ef04fdcaa None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:13.991253 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef53cbc-9356-4316-a043-a2a65525d1c4 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:14.010820 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:14.029664 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70280fe-6fbf-4fc8-bd5a-c816597e4436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:14.030325 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70280fe-6fbf-4fc8-bd5a-c816597e4436 None admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:14.030605 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70280fe-6fbf-4fc8-bd5a-c816597e4436 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:14.030875 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70280fe-6fbf-4fc8-bd5a-c816597e4436 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:14.031624 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70280fe-6fbf-4fc8-bd5a-c816597e4436 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:14.032956 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70280fe-6fbf-4fc8-bd5a-c816597e4436 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:14.048640 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51ecbca-4e44-4850-b720-ee7ef04fdcaa None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:14.065199 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 479/936] 10.222.0.89 () {60 vars in 1144 bytes} [Thu Jul 28 17:32:13 2022] POST /identity/v3/endpoints => generated 347 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:32:14.065199 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1d062c-5d8a-4a1c-93b4-528d5f104251 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:14.065199 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1d062c-5d8a-4a1c-93b4-528d5f104251 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:14.065199 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1d062c-5d8a-4a1c-93b4-528d5f104251 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:14.077815 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-be1d062c-5d8a-4a1c-93b4-528d5f104251 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:32:14.079144 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-be1d062c-5d8a-4a1c-93b4-528d5f104251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:32:14.117379 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:14.126063 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3958f12d-8b0b-4d74-b423-f084578cc8f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:14.127005 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3958f12d-8b0b-4d74-b423-f084578cc8f9 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:14.127950 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3958f12d-8b0b-4d74-b423-f084578cc8f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:14.128247 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3958f12d-8b0b-4d74-b423-f084578cc8f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:14.146735 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3958f12d-8b0b-4d74-b423-f084578cc8f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:14.149242 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3958f12d-8b0b-4d74-b423-f084578cc8f9 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:14.169660 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 480/937] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:32:14 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:32:14.170891 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-3958f12d-8b0b-4d74-b423-f084578cc8f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:32:14.170891 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:32:14.170891 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:32:14.170891 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:32:14.170891 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:32:14.170891 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:32:14.170891 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:32:14.170891 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:32:14.170891 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:32:14.170891 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:32:14.170891 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:32:14.170891 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:32:14.170891 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:32:14.170891 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:32:14.170891 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:32:14.172008 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:32:14.172008 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:32:14.172008 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:32:14.172008 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:32:14.172008 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:32:14.172008 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:32:14.172008 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:32:14.172008 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:32:14.172008 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:32:14.172008 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:32:14.172008 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:32:14.172008 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:32:14.172008 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:32:14.172008 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:32:14.172008 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:32:14.172008 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:32:14.172008 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:32:14.173064 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:32:14.173064 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:32:14.173064 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:32:14.173064 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:32:14.176736 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 458/938] 10.222.0.89 () {56 vars in 1108 bytes} [Thu Jul 28 17:32:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:32:14.177867 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1d062c-5d8a-4a1c-93b4-528d5f104251 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:14.195587 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3958f12d-8b0b-4d74-b423-f084578cc8f9 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:14.302047 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1d062c-5d8a-4a1c-93b4-528d5f104251 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:14.305441 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:14.328067 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3958f12d-8b0b-4d74-b423-f084578cc8f9 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:14.348844 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac7a22a-9cf2-497e-bc47-2f73e53cd0de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:14.350006 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac7a22a-9cf2-497e-bc47-2f73e53cd0de None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:14.350489 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac7a22a-9cf2-497e-bc47-2f73e53cd0de None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:14.350980 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac7a22a-9cf2-497e-bc47-2f73e53cd0de None admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:14.351870 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac7a22a-9cf2-497e-bc47-2f73e53cd0de None admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:32:14.352391 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac7a22a-9cf2-497e-bc47-2f73e53cd0de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:14.354236 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac7a22a-9cf2-497e-bc47-2f73e53cd0de None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:14.368094 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 481/939] 10.222.0.89 () {56 vars in 1109 bytes} [Thu Jul 28 17:32:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:32:14.374685 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeffe7c3-f52d-4218-bbc4-f147ac226475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:14.375337 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac7a22a-9cf2-497e-bc47-2f73e53cd0de None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:14.375828 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeffe7c3-f52d-4218-bbc4-f147ac226475 None admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:14.376306 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeffe7c3-f52d-4218-bbc4-f147ac226475 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:14.376775 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeffe7c3-f52d-4218-bbc4-f147ac226475 None admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:14.377699 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeffe7c3-f52d-4218-bbc4-f147ac226475 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:14.379614 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeffe7c3-f52d-4218-bbc4-f147ac226475 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:14.382539 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: WARNING keystone.server.flask.common [None req-eeffe7c3-f52d-4218-bbc4-f147ac226475 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 28 17:32:14.405704 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.sql.core [None req-eeffe7c3-f52d-4218-bbc4-f147ac226475 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 28 17:32:14.405704 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 28 17:32:14.405704 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [parameters: {'id': '5b92e5c498b246168c75e03b720d4c96', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 28 17:32:14.405704 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80325) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 28 17:32:14.417733 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-eeffe7c3-f52d-4218-bbc4-f147ac226475 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 5b92e5c498b246168c75e03b720d4c96.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 5b92e5c498b246168c75e03b720d4c96. Jul 28 17:32:14.417733 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:32:14.417733 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 28 17:32:14.417733 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 28 17:32:14.417733 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 28 17:32:14.417733 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 28 17:32:14.417733 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 28 17:32:14.417733 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application result = self._query(query) Jul 28 17:32:14.417733 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 28 17:32:14.417733 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application conn.query(q) Jul 28 17:32:14.417733 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 28 17:32:14.417733 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 28 17:32:14.417733 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 28 17:32:14.418599 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application result.read() Jul 28 17:32:14.418599 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 28 17:32:14.418599 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 28 17:32:14.418599 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 28 17:32:14.418599 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 28 17:32:14.418599 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 28 17:32:14.418599 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 28 17:32:14.418599 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 28 17:32:14.418599 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 28 17:32:14.418599 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 28 17:32:14.418599 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:32:14.418599 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 28 17:32:14.418599 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:32:14.418599 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:32:14.418599 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 28 17:32:14.418599 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 28 17:32:14.418599 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 28 17:32:14.418599 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 28 17:32:14.420688 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 28 17:32:14.420688 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application next(self.gen) Jul 28 17:32:14.420688 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 28 17:32:14.420688 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application yield resource Jul 28 17:32:14.420688 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 28 17:32:14.420688 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application next(self.gen) Jul 28 17:32:14.420688 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 28 17:32:14.420688 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application self.session.rollback() Jul 28 17:32:14.420688 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 28 17:32:14.420688 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application self.force_reraise() Jul 28 17:32:14.420688 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 28 17:32:14.420688 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise self.value Jul 28 17:32:14.420688 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 28 17:32:14.420688 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 28 17:32:14.420688 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 28 17:32:14.420688 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application session.commit() Jul 28 17:32:14.420688 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 28 17:32:14.421656 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 28 17:32:14.421656 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 28 17:32:14.421656 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application self._prepare_impl() Jul 28 17:32:14.421656 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 28 17:32:14.421656 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application self.session.flush() Jul 28 17:32:14.421656 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 28 17:32:14.421656 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application self._flush(objects) Jul 28 17:32:14.421656 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 28 17:32:14.421656 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 28 17:32:14.421656 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 28 17:32:14.421656 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application compat.raise_( Jul 28 17:32:14.421656 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 28 17:32:14.421656 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception Jul 28 17:32:14.421656 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 28 17:32:14.421656 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application flush_context.execute() Jul 28 17:32:14.421656 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 28 17:32:14.421656 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rec.execute(self) Jul 28 17:32:14.422569 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 28 17:32:14.422569 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 28 17:32:14.422569 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 28 17:32:14.422569 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 28 17:32:14.422569 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 28 17:32:14.422569 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 28 17:32:14.422569 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 28 17:32:14.422569 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 28 17:32:14.422569 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 28 17:32:14.422569 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 28 17:32:14.422569 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 28 17:32:14.422569 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 28 17:32:14.422569 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 28 17:32:14.422569 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 28 17:32:14.422569 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 28 17:32:14.423351 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 28 17:32:14.423351 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 28 17:32:14.423351 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception Jul 28 17:32:14.423351 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 28 17:32:14.423351 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 28 17:32:14.423351 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 28 17:32:14.423351 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 28 17:32:14.423351 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 28 17:32:14.423351 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application result = self._query(query) Jul 28 17:32:14.423351 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 28 17:32:14.423351 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application conn.query(q) Jul 28 17:32:14.423351 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 28 17:32:14.423351 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 28 17:32:14.423351 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 28 17:32:14.423351 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application result.read() Jul 28 17:32:14.423351 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 28 17:32:14.423351 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 28 17:32:14.425558 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 28 17:32:14.425558 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 28 17:32:14.425558 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 28 17:32:14.425558 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 28 17:32:14.425558 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 28 17:32:14.425558 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 28 17:32:14.425558 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 28 17:32:14.425558 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 28 17:32:14.425558 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [parameters: {'id': '5b92e5c498b246168c75e03b720d4c96', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 28 17:32:14.425558 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 28 17:32:14.425558 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:32:14.425558 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 17:32:14.425558 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:32:14.425558 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:32:14.425558 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 28 17:32:14.426387 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 28 17:32:14.426387 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 28 17:32:14.426387 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 28 17:32:14.426387 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 28 17:32:14.426387 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:32:14.426387 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 17:32:14.426387 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:32:14.426387 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:32:14.426387 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:32:14.426387 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:32:14.426387 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:32:14.426387 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:32:14.426387 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:32:14.426387 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:32:14.426387 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:32:14.426387 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:32:14.426387 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:32:14.427478 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:32:14.427478 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 28 17:32:14.427478 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 28 17:32:14.427478 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:32:14.427478 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:32:14.427478 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 28 17:32:14.427478 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 28 17:32:14.427478 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 5b92e5c498b246168c75e03b720d4c96. Jul 28 17:32:14.427478 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:32:14.427478 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 459/940] 10.222.0.89 () {60 vars in 1140 bytes} [Thu Jul 28 17:32:14 2022] POST /identity/v3/projects => generated 262 bytes in 224 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:32:14.427478 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2befc239-4806-49b9-ba00-858a139a1205 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:14.427478 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2befc239-4806-49b9-ba00-858a139a1205 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:14.428242 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2befc239-4806-49b9-ba00-858a139a1205 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:14.431144 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 460/941] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:32:14 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 17:32:14.432333 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2befc239-4806-49b9-ba00-858a139a1205 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:14.482390 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:14.495091 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac7a22a-9cf2-497e-bc47-2f73e53cd0de None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:14.537781 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0a9af9-6383-4f97-b5ba-e96011d6676c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:14.538827 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0a9af9-6383-4f97-b5ba-e96011d6676c None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:14.539176 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0a9af9-6383-4f97-b5ba-e96011d6676c None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:14.539460 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0a9af9-6383-4f97-b5ba-e96011d6676c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:14.547879 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0a9af9-6383-4f97-b5ba-e96011d6676c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:14.548117 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2befc239-4806-49b9-ba00-858a139a1205 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:14.549352 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0a9af9-6383-4f97-b5ba-e96011d6676c None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:14.563136 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-fd0a9af9-6383-4f97-b5ba-e96011d6676c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 17:32:14.563136 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:32:14.563136 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:32:14.563136 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:32:14.563136 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:32:14.563136 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:32:14.563136 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:32:14.563136 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:32:14.563136 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:32:14.563136 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:32:14.563136 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:32:14.563136 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:32:14.563136 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 17:32:14.563136 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 17:32:14.563136 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 17:32:14.563136 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 17:32:14.564103 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:32:14.564103 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:32:14.564103 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 17:32:14.564103 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:32:14.564103 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 17:32:14.564103 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:32:14.564103 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:32:14.564103 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:32:14.564103 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:32:14.564103 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:32:14.564103 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:32:14.564103 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( Jul 28 17:32:14.564103 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:32:14.564103 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:32:14.564103 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:32:14.564103 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:32:14.564103 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:32:14.564103 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:32:14.565066 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:32:14.565066 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:32:14.565066 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 17:32:14.565066 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 17:32:14.565066 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 17:32:14.565066 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 17:32:14.565066 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 17:32:14.565066 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 17:32:14.565066 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 17:32:14.565066 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:32:14.565066 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 482/942] 10.222.0.89 () {56 vars in 1117 bytes} [Thu Jul 28 17:32:14 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:32:14.565889 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd4133a-b23d-4cd7-bdb2-277895197c1a None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:14.565958 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd4133a-b23d-4cd7-bdb2-277895197c1a None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:14.566435 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd4133a-b23d-4cd7-bdb2-277895197c1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:14.586389 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-7bd4133a-b23d-4cd7-bdb2-277895197c1a None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:32:14.587652 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-7bd4133a-b23d-4cd7-bdb2-277895197c1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:32:14.591289 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86c3441-3ecb-434b-86d7-86cdd7103e33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:14.592048 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86c3441-3ecb-434b-86d7-86cdd7103e33 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:14.592284 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86c3441-3ecb-434b-86d7-86cdd7103e33 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:14.592555 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86c3441-3ecb-434b-86d7-86cdd7103e33 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:14.599656 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86c3441-3ecb-434b-86d7-86cdd7103e33 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:14.601267 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86c3441-3ecb-434b-86d7-86cdd7103e33 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:14.607419 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:14.613124 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-e86c3441-3ecb-434b-86d7-86cdd7103e33 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:32:14.613124 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:32:14.613124 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:32:14.613124 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:32:14.613124 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:32:14.613124 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:32:14.613124 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:32:14.613124 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:32:14.613124 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:32:14.613124 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:32:14.613124 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:32:14.613124 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:32:14.613124 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 17:32:14.613124 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 17:32:14.613124 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 17:32:14.614197 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 17:32:14.614197 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:32:14.614197 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:32:14.614197 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:32:14.614197 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:32:14.614197 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:32:14.614197 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:32:14.614197 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:32:14.614197 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:32:14.614197 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:32:14.614197 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:32:14.614197 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:32:14.614197 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:32:14.614197 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:32:14.614197 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:32:14.614197 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:32:14.614197 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:32:14.615294 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 17:32:14.615294 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 17:32:14.615294 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 17:32:14.615294 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 17:32:14.615294 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 17:32:14.615294 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 17:32:14.615294 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:32:14.615294 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:32:14.619120 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 461/943] 10.222.0.89 () {56 vars in 1111 bytes} [Thu Jul 28 17:32:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:32:14.619955 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86c3441-3ecb-434b-86d7-86cdd7103e33 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:14.677207 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 483/944] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:32:14 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:32:14.677207 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd4133a-b23d-4cd7-bdb2-277895197c1a None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:14.748504 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:14.751647 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86c3441-3ecb-434b-86d7-86cdd7103e33 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:14.786828 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4babfe-45d3-46f1-b0e7-3fcb0bf58e68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:14.787447 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4babfe-45d3-46f1-b0e7-3fcb0bf58e68 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:14.787722 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4babfe-45d3-46f1-b0e7-3fcb0bf58e68 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:14.787990 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4babfe-45d3-46f1-b0e7-3fcb0bf58e68 None admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:14.788773 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4babfe-45d3-46f1-b0e7-3fcb0bf58e68 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:32:14.789078 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4babfe-45d3-46f1-b0e7-3fcb0bf58e68 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:14.790512 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4babfe-45d3-46f1-b0e7-3fcb0bf58e68 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:14.795704 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd4133a-b23d-4cd7-bdb2-277895197c1a None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:14.812680 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 462/945] 10.222.0.89 () {56 vars in 1149 bytes} [Thu Jul 28 17:32:14 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:32:14.812680 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4babfe-45d3-46f1-b0e7-3fcb0bf58e68 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:14.843723 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead4c49f-9ff4-4ce8-ae83-1c73a8324f88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:14.844586 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead4c49f-9ff4-4ce8-ae83-1c73a8324f88 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:14.845052 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead4c49f-9ff4-4ce8-ae83-1c73a8324f88 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:14.845359 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead4c49f-9ff4-4ce8-ae83-1c73a8324f88 None admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:14.846231 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead4c49f-9ff4-4ce8-ae83-1c73a8324f88 None admin] RBAC: Adding query filter params (name=service) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:32:14.846390 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead4c49f-9ff4-4ce8-ae83-1c73a8324f88 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:14.847792 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead4c49f-9ff4-4ce8-ae83-1c73a8324f88 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:14.873545 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 484/946] 10.222.0.89 () {56 vars in 1112 bytes} [Thu Jul 28 17:32:14 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:32:14.874421 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead4c49f-9ff4-4ce8-ae83-1c73a8324f88 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:14.909563 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:14.933249 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4babfe-45d3-46f1-b0e7-3fcb0bf58e68 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:14.987881 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1d152c-f72e-46f7-b5c4-4917dfa4cc44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:14.989672 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1d152c-f72e-46f7-b5c4-4917dfa4cc44 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:14.990147 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1d152c-f72e-46f7-b5c4-4917dfa4cc44 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:14.990630 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1d152c-f72e-46f7-b5c4-4917dfa4cc44 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:14.991729 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1d152c-f72e-46f7-b5c4-4917dfa4cc44 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:14.994286 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1d152c-f72e-46f7-b5c4-4917dfa4cc44 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:15.009362 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-2e1d152c-f72e-46f7-b5c4-4917dfa4cc44 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 17:32:15.009362 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:32:15.009362 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 17:32:15.009362 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 17:32:15.009362 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 17:32:15.009362 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 17:32:15.009362 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 17:32:15.009362 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:32:15.009362 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 17:32:15.009362 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:32:15.009362 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 17:32:15.009362 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:32:15.009362 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:32:15.009362 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:32:15.009362 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:32:15.009362 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:32:15.009362 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:32:15.010806 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:32:15.010806 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:32:15.010806 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:32:15.010806 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:32:15.010806 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:32:15.010806 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:32:15.010806 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 17:32:15.010806 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 17:32:15.010806 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 17:32:15.010806 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 17:32:15.010806 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:32:15.010806 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:32:15.010806 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:32:15.010806 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:32:15.010806 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:32:15.010806 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:32:15.011734 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:32:15.011734 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:32:15.011734 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:32:15.011734 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:32:15.011734 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:32:15.011734 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:32:15.011734 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:32:15.011734 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:32:15.011734 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:32:15.011734 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:32:15.011734 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 17:32:15.011734 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:32:15.011734 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 17:32:15.011734 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:32:15.019596 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 463/947] 10.222.0.89 () {56 vars in 1126 bytes} [Thu Jul 28 17:32:14 2022] GET /identity/v3/services/object-store => generated 93 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:32:15.021114 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1d152c-f72e-46f7-b5c4-4917dfa4cc44 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:15.023916 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead4c49f-9ff4-4ce8-ae83-1c73a8324f88 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:15.048907 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:15.078543 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f712d405-e863-478c-8d96-2b68ea2e1cc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:15.079201 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f712d405-e863-478c-8d96-2b68ea2e1cc7 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:15.080042 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f712d405-e863-478c-8d96-2b68ea2e1cc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:15.080042 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f712d405-e863-478c-8d96-2b68ea2e1cc7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:15.090798 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f712d405-e863-478c-8d96-2b68ea2e1cc7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:15.092319 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f712d405-e863-478c-8d96-2b68ea2e1cc7 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:15.108141 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-f712d405-e863-478c-8d96-2b68ea2e1cc7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:32:15.108141 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:32:15.108141 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:32:15.108141 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:32:15.108141 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:32:15.108141 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:32:15.108141 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:32:15.108141 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:32:15.108141 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:32:15.108141 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:32:15.108141 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:32:15.108141 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:32:15.108141 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:32:15.108141 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:32:15.108141 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:32:15.109489 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:32:15.109489 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:32:15.109489 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:32:15.109489 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:32:15.109489 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:32:15.109489 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:32:15.109489 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:32:15.109489 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:32:15.109489 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( Jul 28 17:32:15.109489 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:32:15.109489 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:32:15.109489 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:32:15.109489 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:32:15.109489 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:32:15.109489 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:32:15.109489 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:32:15.109489 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:32:15.110374 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:32:15.110374 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:32:15.110374 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:32:15.110374 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:32:15.110374 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 485/948] 10.222.0.89 () {56 vars in 1108 bytes} [Thu Jul 28 17:32:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:32:15.115242 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f712d405-e863-478c-8d96-2b68ea2e1cc7 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:15.155617 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1d152c-f72e-46f7-b5c4-4917dfa4cc44 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:15.191320 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3c2d88-bd23-4100-ba47-b8abf2092ed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:15.191946 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3c2d88-bd23-4100-ba47-b8abf2092ed8 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:15.192228 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3c2d88-bd23-4100-ba47-b8abf2092ed8 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:15.192514 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3c2d88-bd23-4100-ba47-b8abf2092ed8 None admin] PATH_INFO: `/v3/services` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:15.192919 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:15.193370 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3c2d88-bd23-4100-ba47-b8abf2092ed8 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:32:15.193550 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3c2d88-bd23-4100-ba47-b8abf2092ed8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:15.194914 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3c2d88-bd23-4100-ba47-b8abf2092ed8 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:15.200722 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 464/949] 10.222.0.89 () {56 vars in 1122 bytes} [Thu Jul 28 17:32:15 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:32:15.207786 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3c2d88-bd23-4100-ba47-b8abf2092ed8 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:15.252016 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f712d405-e863-478c-8d96-2b68ea2e1cc7 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:15.298804 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b25a39-c67a-4551-8d31-2c17943f21e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:15.299820 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b25a39-c67a-4551-8d31-2c17943f21e9 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:15.300457 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b25a39-c67a-4551-8d31-2c17943f21e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:15.300878 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b25a39-c67a-4551-8d31-2c17943f21e9 None admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:15.302434 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b25a39-c67a-4551-8d31-2c17943f21e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:32:15.303012 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b25a39-c67a-4551-8d31-2c17943f21e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:15.304807 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b25a39-c67a-4551-8d31-2c17943f21e9 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:15.321946 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3c2d88-bd23-4100-ba47-b8abf2092ed8 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:15.322194 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 486/950] 10.222.0.89 () {56 vars in 1109 bytes} [Thu Jul 28 17:32:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:32:15.327797 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:15.328568 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b25a39-c67a-4551-8d31-2c17943f21e9 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:15.372653 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63225bc1-1d2c-4b6b-a92e-3ed4f77caeaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:15.375029 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63225bc1-1d2c-4b6b-a92e-3ed4f77caeaa None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:15.375310 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63225bc1-1d2c-4b6b-a92e-3ed4f77caeaa None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:15.375583 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63225bc1-1d2c-4b6b-a92e-3ed4f77caeaa None admin] PATH_INFO: `/v3/services` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:15.376270 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63225bc1-1d2c-4b6b-a92e-3ed4f77caeaa None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:32:15.376570 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63225bc1-1d2c-4b6b-a92e-3ed4f77caeaa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:15.378002 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63225bc1-1d2c-4b6b-a92e-3ed4f77caeaa None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:15.383700 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 465/951] 10.222.0.89 () {56 vars in 1122 bytes} [Thu Jul 28 17:32:15 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:32:15.391396 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63225bc1-1d2c-4b6b-a92e-3ed4f77caeaa None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:15.469062 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b25a39-c67a-4551-8d31-2c17943f21e9 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:15.506211 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:15.512811 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63225bc1-1d2c-4b6b-a92e-3ed4f77caeaa None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:15.519499 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef97e728-4b9a-4bbd-9fcc-17eac3544778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:15.520318 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef97e728-4b9a-4bbd-9fcc-17eac3544778 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:15.520777 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef97e728-4b9a-4bbd-9fcc-17eac3544778 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:15.521822 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef97e728-4b9a-4bbd-9fcc-17eac3544778 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:15.532304 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef97e728-4b9a-4bbd-9fcc-17eac3544778 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:15.534370 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef97e728-4b9a-4bbd-9fcc-17eac3544778 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:15.550645 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f030534e-fe11-4be3-8842-b47f72d83a89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:15.550833 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-ef97e728-4b9a-4bbd-9fcc-17eac3544778 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:32:15.550833 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:32:15.550833 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:32:15.550833 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:32:15.550833 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:32:15.550833 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:32:15.550833 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:32:15.550833 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:32:15.550833 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:32:15.550833 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:32:15.550833 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:32:15.550833 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:32:15.550833 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 17:32:15.550833 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 17:32:15.550833 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 17:32:15.551893 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 17:32:15.551893 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:32:15.551893 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:32:15.551893 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:32:15.551893 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:32:15.551893 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:32:15.551893 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:32:15.551893 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:32:15.551893 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( Jul 28 17:32:15.551893 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:32:15.551893 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:32:15.551893 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:32:15.551893 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:32:15.551893 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:32:15.551893 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:32:15.551893 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:32:15.551893 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:32:15.553024 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 17:32:15.553024 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 17:32:15.553024 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 17:32:15.553024 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 17:32:15.553024 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 17:32:15.553024 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 17:32:15.553024 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:32:15.553024 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:32:15.553667 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f030534e-fe11-4be3-8842-b47f72d83a89 None admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:15.553667 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f030534e-fe11-4be3-8842-b47f72d83a89 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:15.553667 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f030534e-fe11-4be3-8842-b47f72d83a89 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:15.554497 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f030534e-fe11-4be3-8842-b47f72d83a89 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:15.560039 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f030534e-fe11-4be3-8842-b47f72d83a89 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:15.560504 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 487/952] 10.222.0.89 () {56 vars in 1111 bytes} [Thu Jul 28 17:32:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:32:15.561765 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef97e728-4b9a-4bbd-9fcc-17eac3544778 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:15.589278 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 466/953] 10.222.0.89 () {60 vars in 1144 bytes} [Thu Jul 28 17:32:15 2022] POST /identity/v3/endpoints => generated 349 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:32:15.687747 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef97e728-4b9a-4bbd-9fcc-17eac3544778 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:15.735385 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dacaf78-5d6e-4f02-8154-7af94c410a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:15.736306 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dacaf78-5d6e-4f02-8154-7af94c410a94 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:15.736736 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dacaf78-5d6e-4f02-8154-7af94c410a94 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:15.737199 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dacaf78-5d6e-4f02-8154-7af94c410a94 None admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:15.738136 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dacaf78-5d6e-4f02-8154-7af94c410a94 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:32:15.738639 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dacaf78-5d6e-4f02-8154-7af94c410a94 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:15.740232 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dacaf78-5d6e-4f02-8154-7af94c410a94 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:15.754801 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 488/954] 10.222.0.89 () {56 vars in 1148 bytes} [Thu Jul 28 17:32:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:32:15.760934 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f030534e-fe11-4be3-8842-b47f72d83a89 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:15.772061 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:15.880531 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f030534e-fe11-4be3-8842-b47f72d83a89 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:15.917128 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794af20e-f2cc-4c8d-adfa-ad71cb80f5e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:15.917841 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794af20e-f2cc-4c8d-adfa-ad71cb80f5e3 None admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:15.918125 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794af20e-f2cc-4c8d-adfa-ad71cb80f5e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:15.918401 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794af20e-f2cc-4c8d-adfa-ad71cb80f5e3 None admin] PATH_INFO: `/v3/projects/04434e8535a64044996316f1d17dc0bc/users/c44387d13a694ee6bb47a13906003330/roles/af4b58a9c88d4cb4861277524ab3e1ed` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:15.920798 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794af20e-f2cc-4c8d-adfa-ad71cb80f5e3 None admin] RBAC: Authorizing `identity:create_grant(project_id=04434e8535a64044996316f1d17dc0bc, user_id=c44387d13a694ee6bb47a13906003330, role_id=af4b58a9c88d4cb4861277524ab3e1ed)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:15.922273 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794af20e-f2cc-4c8d-adfa-ad71cb80f5e3 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:15.940893 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 467/955] 10.222.0.89 () {58 vars in 1439 bytes} [Thu Jul 28 17:32:15 2022] PUT /identity/v3/projects/04434e8535a64044996316f1d17dc0bc/users/c44387d13a694ee6bb47a13906003330/roles/af4b58a9c88d4cb4861277524ab3e1ed => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:32:16.238212 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2db58d5-22d0-452c-94f5-bc0478be2e4a None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:16.238756 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2db58d5-22d0-452c-94f5-bc0478be2e4a None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:16.238756 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2db58d5-22d0-452c-94f5-bc0478be2e4a None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:16.240361 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 489/956] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:32:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:32:16.245625 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8932bf80-8ace-41a4-bc42-62f64ac2c618 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:16.246286 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8932bf80-8ace-41a4-bc42-62f64ac2c618 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:16.246467 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8932bf80-8ace-41a4-bc42-62f64ac2c618 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:16.265596 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-8932bf80-8ace-41a4-bc42-62f64ac2c618 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:32:16.267129 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-8932bf80-8ace-41a4-bc42-62f64ac2c618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:32:16.333258 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:16.343700 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 468/957] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:32:16 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:32:16.365380 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2db58d5-22d0-452c-94f5-bc0478be2e4a None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:16.412843 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:16.476670 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2db58d5-22d0-452c-94f5-bc0478be2e4a None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:16.517786 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516ae0cd-a766-43e0-bca3-5d2926566ee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:16.518345 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516ae0cd-a766-43e0-bca3-5d2926566ee0 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:16.518622 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516ae0cd-a766-43e0-bca3-5d2926566ee0 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:16.518895 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516ae0cd-a766-43e0-bca3-5d2926566ee0 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:16.519522 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516ae0cd-a766-43e0-bca3-5d2926566ee0 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:16.520897 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516ae0cd-a766-43e0-bca3-5d2926566ee0 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:16.530127 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-516ae0cd-a766-43e0-bca3-5d2926566ee0 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 17:32:16.530127 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:32:16.530127 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 17:32:16.530127 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 17:32:16.530127 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 17:32:16.530127 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 17:32:16.530127 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 17:32:16.530127 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:32:16.530127 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 17:32:16.530127 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:32:16.530127 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 17:32:16.530127 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:32:16.530127 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:32:16.530127 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:32:16.530127 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:32:16.530127 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:32:16.530127 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:32:16.531646 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:32:16.531646 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:32:16.531646 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:32:16.531646 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:32:16.531646 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:32:16.531646 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:32:16.531646 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 17:32:16.531646 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 17:32:16.531646 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 17:32:16.531646 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 17:32:16.531646 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:32:16.531646 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:32:16.531646 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:32:16.531646 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:32:16.531646 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:32:16.531646 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:32:16.533208 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:32:16.533208 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( Jul 28 17:32:16.533208 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:32:16.533208 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:32:16.533208 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:32:16.533208 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:32:16.533208 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:32:16.533208 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:32:16.533208 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:32:16.533208 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:32:16.533208 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 17:32:16.533208 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:32:16.533208 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 17:32:16.533208 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:32:16.533208 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 490/958] 10.222.0.89 () {56 vars in 1111 bytes} [Thu Jul 28 17:32:16 2022] GET /identity/v3/services/compute => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:32:16.537724 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8932bf80-8ace-41a4-bc42-62f64ac2c618 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:16.640895 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8932bf80-8ace-41a4-bc42-62f64ac2c618 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:16.677525 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e0ca15-ccc6-429e-b65f-f15246031475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:16.678152 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e0ca15-ccc6-429e-b65f-f15246031475 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:16.678435 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e0ca15-ccc6-429e-b65f-f15246031475 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:16.678709 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e0ca15-ccc6-429e-b65f-f15246031475 None admin] PATH_INFO: `/v3/services` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:16.679397 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e0ca15-ccc6-429e-b65f-f15246031475 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:32:16.679707 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e0ca15-ccc6-429e-b65f-f15246031475 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:16.681077 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e0ca15-ccc6-429e-b65f-f15246031475 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:16.686955 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 469/959] 10.222.0.89 () {56 vars in 1112 bytes} [Thu Jul 28 17:32:16 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:32:16.692487 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516ae0cd-a766-43e0-bca3-5d2926566ee0 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:16.806951 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516ae0cd-a766-43e0-bca3-5d2926566ee0 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:16.817571 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:16.850264 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ff9a04-97ce-42b2-a8fe-4fa0ee2e6ac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:16.850939 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ff9a04-97ce-42b2-a8fe-4fa0ee2e6ac5 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:16.851223 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ff9a04-97ce-42b2-a8fe-4fa0ee2e6ac5 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:16.851495 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ff9a04-97ce-42b2-a8fe-4fa0ee2e6ac5 None admin] PATH_INFO: `/v3/services` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:16.852223 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ff9a04-97ce-42b2-a8fe-4fa0ee2e6ac5 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:32:16.852527 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ff9a04-97ce-42b2-a8fe-4fa0ee2e6ac5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:16.853923 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ff9a04-97ce-42b2-a8fe-4fa0ee2e6ac5 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:16.859724 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 491/960] 10.222.0.89 () {56 vars in 1112 bytes} [Thu Jul 28 17:32:16 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:32:17.086083 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04e26c6-47f0-4b15-9a1d-dea856b4ebe2 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:17.086598 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04e26c6-47f0-4b15-9a1d-dea856b4ebe2 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:17.087029 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04e26c6-47f0-4b15-9a1d-dea856b4ebe2 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:17.088784 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 470/961] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:32:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:32:17.094418 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b76569-adae-4286-9fac-081b879089a1 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:17.095005 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b76569-adae-4286-9fac-081b879089a1 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:17.095494 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b76569-adae-4286-9fac-081b879089a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:17.114486 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-e9b76569-adae-4286-9fac-081b879089a1 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:32:17.116061 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-e9b76569-adae-4286-9fac-081b879089a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:32:17.201637 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 492/962] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:32:17 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 17:32:17.221015 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04e26c6-47f0-4b15-9a1d-dea856b4ebe2 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:17.284129 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:17.327245 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04e26c6-47f0-4b15-9a1d-dea856b4ebe2 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:17.364165 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eea339-b401-4d27-9618-eae93b74ad73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:17.364709 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eea339-b401-4d27-9618-eae93b74ad73 None admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:17.364987 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eea339-b401-4d27-9618-eae93b74ad73 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:17.365291 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eea339-b401-4d27-9618-eae93b74ad73 None admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:17.365976 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1eea339-b401-4d27-9618-eae93b74ad73 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:17.367398 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1eea339-b401-4d27-9618-eae93b74ad73 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:17.387488 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.sql.core [None req-b1eea339-b401-4d27-9618-eae93b74ad73 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 28 17:32:17.387488 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 28 17:32:17.387488 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [parameters: {'user_id': '4fb182dd792947e5a7acee072314768e', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 28 17:32:17.387488 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80325) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 28 17:32:17.394672 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-b1eea339-b401-4d27-9618-eae93b74ad73 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 28 17:32:17.394672 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:32:17.394672 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 28 17:32:17.394672 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 28 17:32:17.394672 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 28 17:32:17.394672 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 28 17:32:17.394672 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 28 17:32:17.394672 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application result = self._query(query) Jul 28 17:32:17.394672 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 28 17:32:17.394672 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application conn.query(q) Jul 28 17:32:17.394672 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 28 17:32:17.394672 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 28 17:32:17.394672 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 28 17:32:17.394672 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application result.read() Jul 28 17:32:17.394672 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 28 17:32:17.395560 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 28 17:32:17.395560 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 28 17:32:17.395560 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 28 17:32:17.395560 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 28 17:32:17.395560 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 28 17:32:17.395560 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 28 17:32:17.395560 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 28 17:32:17.395560 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 28 17:32:17.395560 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:32:17.395560 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 28 17:32:17.395560 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:32:17.395560 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:32:17.395560 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 28 17:32:17.395560 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 28 17:32:17.395560 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 28 17:32:17.395560 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 28 17:32:17.395560 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 28 17:32:17.395560 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application next(self.gen) Jul 28 17:32:17.396615 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 28 17:32:17.396615 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application yield resource Jul 28 17:32:17.396615 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 28 17:32:17.396615 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application next(self.gen) Jul 28 17:32:17.396615 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 28 17:32:17.396615 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application self.session.rollback() Jul 28 17:32:17.396615 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 28 17:32:17.396615 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application self.force_reraise() Jul 28 17:32:17.396615 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 28 17:32:17.396615 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise self.value Jul 28 17:32:17.396615 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 28 17:32:17.396615 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 28 17:32:17.396615 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 28 17:32:17.396615 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application session.commit() Jul 28 17:32:17.396615 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 28 17:32:17.396615 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 28 17:32:17.396615 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 28 17:32:17.398253 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application self._prepare_impl() Jul 28 17:32:17.398253 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 28 17:32:17.398253 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application self.session.flush() Jul 28 17:32:17.398253 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 28 17:32:17.398253 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application self._flush(objects) Jul 28 17:32:17.398253 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 28 17:32:17.398253 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 28 17:32:17.398253 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 28 17:32:17.398253 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application compat.raise_( Jul 28 17:32:17.398253 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 28 17:32:17.398253 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception Jul 28 17:32:17.398253 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 28 17:32:17.398253 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application flush_context.execute() Jul 28 17:32:17.398253 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 28 17:32:17.398253 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rec.execute(self) Jul 28 17:32:17.398253 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 28 17:32:17.398253 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 28 17:32:17.399167 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 28 17:32:17.399167 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 28 17:32:17.399167 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 28 17:32:17.399167 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 28 17:32:17.399167 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 28 17:32:17.399167 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 28 17:32:17.399167 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 28 17:32:17.399167 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 28 17:32:17.399167 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 28 17:32:17.399167 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 28 17:32:17.399167 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 28 17:32:17.399167 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 28 17:32:17.399167 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 28 17:32:17.399167 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 28 17:32:17.399167 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 28 17:32:17.399167 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception Jul 28 17:32:17.400013 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 28 17:32:17.400013 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 28 17:32:17.400013 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 28 17:32:17.400013 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 28 17:32:17.400013 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 28 17:32:17.400013 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application result = self._query(query) Jul 28 17:32:17.400013 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 28 17:32:17.400013 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application conn.query(q) Jul 28 17:32:17.400013 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 28 17:32:17.400013 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 28 17:32:17.400013 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 28 17:32:17.400013 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application result.read() Jul 28 17:32:17.400013 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 28 17:32:17.400013 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 28 17:32:17.400013 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 28 17:32:17.400013 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 28 17:32:17.403465 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 28 17:32:17.403465 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 28 17:32:17.403465 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 28 17:32:17.403465 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 28 17:32:17.403465 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 28 17:32:17.403465 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 28 17:32:17.403465 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [parameters: {'user_id': '4fb182dd792947e5a7acee072314768e', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 28 17:32:17.403465 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 28 17:32:17.403465 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:32:17.403465 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 17:32:17.403465 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:32:17.403465 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:32:17.403465 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:32:17.403465 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:32:17.403465 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:32:17.406063 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:32:17.406063 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:32:17.406063 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:32:17.406063 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:32:17.406063 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:32:17.406063 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:32:17.406063 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:32:17.406063 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 28 17:32:17.406063 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 28 17:32:17.406063 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:32:17.406063 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:32:17.406063 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 17:32:17.406063 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:32:17.406063 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 17:32:17.406063 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:32:17.406063 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 28 17:32:17.406063 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 28 17:32:17.408259 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 28 17:32:17.408259 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 28 17:32:17.408259 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 28 17:32:17.408259 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 28 17:32:17.408259 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 28 17:32:17.408259 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:32:17.408259 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 471/963] 10.222.0.89 () {60 vars in 1131 bytes} [Thu Jul 28 17:32:17 2022] POST /identity/v3/users => generated 160 bytes in 176 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:32:17.409215 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b76569-adae-4286-9fac-081b879089a1 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:17.510413 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:17.517198 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b76569-adae-4286-9fac-081b879089a1 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:17.549770 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2eb0f2-f20c-4efe-a815-fa4db6849a7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:17.550414 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2eb0f2-f20c-4efe-a815-fa4db6849a7a None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:17.550743 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2eb0f2-f20c-4efe-a815-fa4db6849a7a None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:17.551023 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2eb0f2-f20c-4efe-a815-fa4db6849a7a None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:17.558407 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2eb0f2-f20c-4efe-a815-fa4db6849a7a None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:17.559861 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2eb0f2-f20c-4efe-a815-fa4db6849a7a None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:17.572851 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-ab2eb0f2-f20c-4efe-a815-fa4db6849a7a None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 28 17:32:17.572851 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:32:17.572851 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:32:17.572851 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:32:17.572851 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:32:17.572851 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:32:17.572851 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:32:17.572851 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:32:17.572851 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:32:17.572851 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:32:17.572851 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:32:17.572851 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:32:17.572851 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 17:32:17.572851 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 17:32:17.572851 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 17:32:17.572851 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 17:32:17.573865 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:32:17.573865 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:32:17.573865 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 17:32:17.573865 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:32:17.573865 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 17:32:17.573865 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:32:17.573865 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:32:17.573865 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:32:17.573865 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:32:17.573865 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:32:17.573865 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:32:17.573865 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( Jul 28 17:32:17.573865 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:32:17.573865 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:32:17.573865 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:32:17.573865 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:32:17.573865 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:32:17.573865 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:32:17.575241 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:32:17.575241 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:32:17.575241 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 17:32:17.575241 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 17:32:17.575241 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 17:32:17.575241 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 17:32:17.575241 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 17:32:17.575241 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 17:32:17.575241 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 28 17:32:17.575241 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:32:17.575241 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 493/964] 10.222.0.89 () {56 vars in 1099 bytes} [Thu Jul 28 17:32:17 2022] GET /identity/v3/users/cinder => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:32:17.582809 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eea339-b401-4d27-9618-eae93b74ad73 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:17.691749 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eea339-b401-4d27-9618-eae93b74ad73 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:17.716414 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:17.729097 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ec257a-b936-4bf5-be56-55d76ce40c12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:17.729970 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ec257a-b936-4bf5-be56-55d76ce40c12 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:17.730682 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ec257a-b936-4bf5-be56-55d76ce40c12 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:17.730963 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ec257a-b936-4bf5-be56-55d76ce40c12 None admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:17.731743 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ec257a-b936-4bf5-be56-55d76ce40c12 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:32:17.733106 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ec257a-b936-4bf5-be56-55d76ce40c12 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:17.735129 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ec257a-b936-4bf5-be56-55d76ce40c12 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:17.781360 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 472/965] 10.222.0.89 () {56 vars in 1101 bytes} [Thu Jul 28 17:32:17 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:32:17.839376 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f9c5f2-3750-48fc-a793-841b01e89c84 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:17.839665 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f9c5f2-3750-48fc-a793-841b01e89c84 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:17.839940 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f9c5f2-3750-48fc-a793-841b01e89c84 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:17.841775 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 494/966] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:32:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:32:17.848707 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0595e9c6-5434-49ee-972e-d48d56f15b8e None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:17.849000 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0595e9c6-5434-49ee-972e-d48d56f15b8e None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:17.849406 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0595e9c6-5434-49ee-972e-d48d56f15b8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:17.869739 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-0595e9c6-5434-49ee-972e-d48d56f15b8e None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:32:17.871628 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-0595e9c6-5434-49ee-972e-d48d56f15b8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:32:17.954661 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 473/967] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:32:17 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:32:17.978480 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f9c5f2-3750-48fc-a793-841b01e89c84 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:18.095039 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f9c5f2-3750-48fc-a793-841b01e89c84 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:18.131802 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebb1ed4-f15a-4866-a982-ba44070be636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:18.132436 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebb1ed4-f15a-4866-a982-ba44070be636 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:18.132711 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebb1ed4-f15a-4866-a982-ba44070be636 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:18.132980 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebb1ed4-f15a-4866-a982-ba44070be636 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:18.133626 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebb1ed4-f15a-4866-a982-ba44070be636 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:18.134976 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebb1ed4-f15a-4866-a982-ba44070be636 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:18.145193 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-3ebb1ed4-f15a-4866-a982-ba44070be636 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 17:32:18.145193 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:32:18.145193 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 17:32:18.145193 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 17:32:18.145193 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 17:32:18.145193 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 17:32:18.145193 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 17:32:18.145193 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:32:18.145193 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 17:32:18.145193 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:32:18.145193 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 17:32:18.145193 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:32:18.145193 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:32:18.145193 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:32:18.145193 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:32:18.145193 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:32:18.145193 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:32:18.146193 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:32:18.146193 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:32:18.146193 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:32:18.146193 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:32:18.146193 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:32:18.146193 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:32:18.146193 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 17:32:18.146193 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 17:32:18.146193 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 17:32:18.146193 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 17:32:18.146193 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:32:18.146193 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:32:18.146193 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:32:18.146193 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:32:18.146193 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:32:18.146193 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:32:18.147040 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:32:18.147040 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( Jul 28 17:32:18.147040 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:32:18.147040 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:32:18.147040 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:32:18.147040 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:32:18.147040 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:32:18.147040 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:32:18.147040 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:32:18.147040 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:32:18.147040 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 17:32:18.147040 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:32:18.147040 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 17:32:18.147040 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:32:18.147040 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 495/968] 10.222.0.89 () {56 vars in 1126 bytes} [Thu Jul 28 17:32:17 2022] GET /identity/v3/services/object-store => generated 93 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:32:18.152715 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0595e9c6-5434-49ee-972e-d48d56f15b8e None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:18.158021 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23b55f4-0cdc-4fd2-91b5-7c4ed4d46127 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:18.158312 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23b55f4-0cdc-4fd2-91b5-7c4ed4d46127 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:18.159161 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23b55f4-0cdc-4fd2-91b5-7c4ed4d46127 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:18.160677 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 496/969] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:32:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 17:32:18.165544 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa74063-a3bc-4f13-ae54-4f2b69d7b93c None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:18.165831 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa74063-a3bc-4f13-ae54-4f2b69d7b93c None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:18.166104 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa74063-a3bc-4f13-ae54-4f2b69d7b93c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:18.184591 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-7aa74063-a3bc-4f13-ae54-4f2b69d7b93c None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:32:18.185848 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-7aa74063-a3bc-4f13-ae54-4f2b69d7b93c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:32:18.205776 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:18.259763 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 497/970] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:32:18 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 17:32:18.263328 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0595e9c6-5434-49ee-972e-d48d56f15b8e None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:18.275594 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:18.283634 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa74063-a3bc-4f13-ae54-4f2b69d7b93c None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:18.302686 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a770738d-98fd-419b-a319-a90820e41dd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:18.303905 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a770738d-98fd-419b-a319-a90820e41dd6 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:18.304187 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a770738d-98fd-419b-a319-a90820e41dd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:18.304457 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a770738d-98fd-419b-a319-a90820e41dd6 None admin] PATH_INFO: `/v3/services` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:18.305141 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a770738d-98fd-419b-a319-a90820e41dd6 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:32:18.305485 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a770738d-98fd-419b-a319-a90820e41dd6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:18.306842 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a770738d-98fd-419b-a319-a90820e41dd6 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:18.312697 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 474/971] 10.222.0.89 () {56 vars in 1122 bytes} [Thu Jul 28 17:32:18 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:32:18.318397 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a770738d-98fd-419b-a319-a90820e41dd6 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:18.396837 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa74063-a3bc-4f13-ae54-4f2b69d7b93c None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:18.425399 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a770738d-98fd-419b-a319-a90820e41dd6 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:18.433959 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a33ae1-47f9-45f3-b507-9688ec7371cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:18.434598 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a33ae1-47f9-45f3-b507-9688ec7371cb None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:18.434868 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a33ae1-47f9-45f3-b507-9688ec7371cb None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:18.435144 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a33ae1-47f9-45f3-b507-9688ec7371cb None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:18.446317 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a33ae1-47f9-45f3-b507-9688ec7371cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:32:18.451812 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-e6a33ae1-47f9-45f3-b507-9688ec7371cb None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 17:32:18.451812 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:32:18.451812 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:32:18.451812 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:32:18.451812 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:32:18.451812 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:32:18.451812 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:32:18.451812 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:32:18.451812 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:32:18.451812 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:32:18.451812 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:32:18.451812 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:32:18.451812 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:32:18.451812 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:32:18.451812 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:32:18.451812 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:32:18.452750 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:32:18.452750 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:32:18.452750 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:32:18.452750 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:32:18.452750 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:32:18.452750 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:32:18.452750 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:32:18.452750 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:32:18.452750 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:32:18.452750 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:32:18.452750 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:32:18.452750 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( Jul 28 17:32:18.452750 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:32:18.452750 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:32:18.452750 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:32:18.452750 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:32:18.454243 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:32:18.454243 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:32:18.454243 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:32:18.454243 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:32:18.454243 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:32:18.454243 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:32:18.454243 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:32:18.454243 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:32:18.454243 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:32:18.454243 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:32:18.454243 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 17:32:18.454243 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:32:18.454243 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 498/972] 10.222.0.89 () {56 vars in 1120 bytes} [Thu Jul 28 17:32:18 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:32:18.459739 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a33ae1-47f9-45f3-b507-9688ec7371cb None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:18.467049 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf87858-9d76-4567-9b09-b9a75e9cb94e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:18.467960 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf87858-9d76-4567-9b09-b9a75e9cb94e None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:18.468075 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf87858-9d76-4567-9b09-b9a75e9cb94e None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:18.468329 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf87858-9d76-4567-9b09-b9a75e9cb94e None admin] PATH_INFO: `/v3/services` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:18.469018 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf87858-9d76-4567-9b09-b9a75e9cb94e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:32:18.469357 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf87858-9d76-4567-9b09-b9a75e9cb94e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:18.470838 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf87858-9d76-4567-9b09-b9a75e9cb94e None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:18.477007 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:18.481615 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 475/973] 10.222.0.89 () {56 vars in 1122 bytes} [Thu Jul 28 17:32:18 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:32:18.482976 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf87858-9d76-4567-9b09-b9a75e9cb94e None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:18.529675 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:18.570094 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a33ae1-47f9-45f3-b507-9688ec7371cb None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:18.597382 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf87858-9d76-4567-9b09-b9a75e9cb94e None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:18.605529 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b609bf4f-1411-4f0b-990c-f6313bbd21cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:18.606150 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b609bf4f-1411-4f0b-990c-f6313bbd21cc None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:18.606434 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b609bf4f-1411-4f0b-990c-f6313bbd21cc None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:18.606701 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b609bf4f-1411-4f0b-990c-f6313bbd21cc None admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:18.607936 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b609bf4f-1411-4f0b-990c-f6313bbd21cc None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:32:18.608276 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b609bf4f-1411-4f0b-990c-f6313bbd21cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:18.609705 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b609bf4f-1411-4f0b-990c-f6313bbd21cc None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:18.618216 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 499/974] 10.222.0.89 () {56 vars in 1115 bytes} [Thu Jul 28 17:32:18 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:32:18.624051 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b609bf4f-1411-4f0b-990c-f6313bbd21cc None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:18.636556 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e731abe6-7e19-4695-b508-dc6477fabeff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:18.637522 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e731abe6-7e19-4695-b508-dc6477fabeff None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:18.637960 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e731abe6-7e19-4695-b508-dc6477fabeff None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:18.638233 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e731abe6-7e19-4695-b508-dc6477fabeff None admin] PATH_INFO: `/v3/endpoints` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:18.639597 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e731abe6-7e19-4695-b508-dc6477fabeff None admin] RBAC: Adding query filter params (interface=admin, service_id=1edcf516a5e14b8ca7e79aa7abe3f823, region_id=RegionOne) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:32:18.639908 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e731abe6-7e19-4695-b508-dc6477fabeff None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:18.641631 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e731abe6-7e19-4695-b508-dc6477fabeff None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:18.651916 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 476/975] 10.222.0.89 () {56 vars in 1249 bytes} [Thu Jul 28 17:32:18 2022] GET /identity/v3/endpoints?service_id=1edcf516a5e14b8ca7e79aa7abe3f823&interface=admin®ion_id=RegionOne => generated 193 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:32:18.653122 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e731abe6-7e19-4695-b508-dc6477fabeff None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:18.720381 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:18.747244 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b609bf4f-1411-4f0b-990c-f6313bbd21cc None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:18.760058 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e731abe6-7e19-4695-b508-dc6477fabeff None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:18.761896 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:18.794594 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97dd82f-d231-46d8-b074-57939c3c262a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:18.796052 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97dd82f-d231-46d8-b074-57939c3c262a None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:18.796338 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97dd82f-d231-46d8-b074-57939c3c262a None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:18.796709 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97dd82f-d231-46d8-b074-57939c3c262a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:18.803394 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d3d7b6-2bf8-4734-8b47-24affda3cd9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:18.804030 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d3d7b6-2bf8-4734-8b47-24affda3cd9d None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:18.804310 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d3d7b6-2bf8-4734-8b47-24affda3cd9d None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:18.804581 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d3d7b6-2bf8-4734-8b47-24affda3cd9d None admin] PATH_INFO: `/v3/services` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:18.805426 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d3d7b6-2bf8-4734-8b47-24affda3cd9d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:18.806808 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d3d7b6-2bf8-4734-8b47-24affda3cd9d None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:18.814156 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97dd82f-d231-46d8-b074-57939c3c262a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:18.825421 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 477/976] 10.222.0.89 () {56 vars in 1087 bytes} [Thu Jul 28 17:32:18 2022] GET /identity/v3/services => generated 1497 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:32:18.825645 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97dd82f-d231-46d8-b074-57939c3c262a None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:18.835340 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-b97dd82f-d231-46d8-b074-57939c3c262a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:32:18.835340 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:32:18.835340 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:32:18.835340 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:32:18.835340 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:32:18.835340 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:32:18.835340 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:32:18.835340 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:32:18.835340 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:32:18.835340 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:32:18.835340 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:32:18.835340 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:32:18.835340 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:32:18.835340 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:32:18.835340 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:32:18.836339 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:32:18.836339 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:32:18.836339 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:32:18.836339 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:32:18.836339 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:32:18.836339 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:32:18.836339 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:32:18.836339 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:32:18.836339 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( Jul 28 17:32:18.836339 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:32:18.836339 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:32:18.836339 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:32:18.836339 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:32:18.836339 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:32:18.836339 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:32:18.836339 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:32:18.836339 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:32:18.837262 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:32:18.837262 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:32:18.837262 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:32:18.837262 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:32:18.837262 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 500/977] 10.222.0.89 () {56 vars in 1108 bytes} [Thu Jul 28 17:32:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:32:18.844686 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d3d7b6-2bf8-4734-8b47-24affda3cd9d None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:18.957498 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d3d7b6-2bf8-4734-8b47-24affda3cd9d None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:18.984109 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e428827-68dc-43e3-9d12-d5cbd2202232 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:18.984704 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e428827-68dc-43e3-9d12-d5cbd2202232 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:18.984982 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e428827-68dc-43e3-9d12-d5cbd2202232 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:18.990596 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 501/978] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:32:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:32:18.991859 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5749fab2-6284-4f24-931b-544d0f562814 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:18.992259 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5749fab2-6284-4f24-931b-544d0f562814 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:18.992543 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5749fab2-6284-4f24-931b-544d0f562814 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:18.995479 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9034d9-1ac2-483b-94a2-a6e0383d0e6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:18.996353 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9034d9-1ac2-483b-94a2-a6e0383d0e6b None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:18.996633 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9034d9-1ac2-483b-94a2-a6e0383d0e6b None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:18.996908 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9034d9-1ac2-483b-94a2-a6e0383d0e6b None admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:18.997637 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9034d9-1ac2-483b-94a2-a6e0383d0e6b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:32:18.998466 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9034d9-1ac2-483b-94a2-a6e0383d0e6b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:18.999896 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9034d9-1ac2-483b-94a2-a6e0383d0e6b None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:19.012484 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 478/979] 10.222.0.89 () {56 vars in 1109 bytes} [Thu Jul 28 17:32:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:32:19.016358 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-5749fab2-6284-4f24-931b-544d0f562814 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:32:19.017781 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-5749fab2-6284-4f24-931b-544d0f562814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:32:19.018893 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9034d9-1ac2-483b-94a2-a6e0383d0e6b None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:19.050177 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:19.092758 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 502/980] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:32:18 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:32:19.115925 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5749fab2-6284-4f24-931b-544d0f562814 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:19.129947 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9034d9-1ac2-483b-94a2-a6e0383d0e6b None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:19.166145 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196076b6-7264-4b66-b745-16aee0ebe7d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:19.166795 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196076b6-7264-4b66-b745-16aee0ebe7d9 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:19.167079 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196076b6-7264-4b66-b745-16aee0ebe7d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:19.167354 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196076b6-7264-4b66-b745-16aee0ebe7d9 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:19.176160 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196076b6-7264-4b66-b745-16aee0ebe7d9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:19.177643 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196076b6-7264-4b66-b745-16aee0ebe7d9 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:19.182976 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:19.190349 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-196076b6-7264-4b66-b745-16aee0ebe7d9 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 17:32:19.190349 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:32:19.190349 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:32:19.190349 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:32:19.190349 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:32:19.190349 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:32:19.190349 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:32:19.190349 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:32:19.190349 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:32:19.190349 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:32:19.190349 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:32:19.190349 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:32:19.190349 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 17:32:19.190349 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 17:32:19.190349 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 17:32:19.190349 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 17:32:19.191338 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:32:19.191338 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:32:19.191338 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 17:32:19.191338 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:32:19.191338 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 17:32:19.191338 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:32:19.191338 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:32:19.191338 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:32:19.191338 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:32:19.191338 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:32:19.191338 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:32:19.191338 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:32:19.191338 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:32:19.191338 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:32:19.191338 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:32:19.191338 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:32:19.191338 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:32:19.191338 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:32:19.192279 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:32:19.192279 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:32:19.192279 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 17:32:19.192279 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 17:32:19.192279 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 17:32:19.192279 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 17:32:19.192279 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 17:32:19.192279 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 17:32:19.192279 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 17:32:19.192279 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:32:19.192279 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 479/981] 10.222.0.89 () {56 vars in 1117 bytes} [Thu Jul 28 17:32:19 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:32:19.197106 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196076b6-7264-4b66-b745-16aee0ebe7d9 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:19.232183 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5749fab2-6284-4f24-931b-544d0f562814 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:19.271835 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7e41af-9493-463b-88bc-0038f8c6978b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:19.272385 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7e41af-9493-463b-88bc-0038f8c6978b None admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:19.272660 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7e41af-9493-463b-88bc-0038f8c6978b None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:19.272926 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7e41af-9493-463b-88bc-0038f8c6978b None admin] PATH_INFO: `/v3/services` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:19.273663 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7e41af-9493-463b-88bc-0038f8c6978b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:19.275019 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7e41af-9493-463b-88bc-0038f8c6978b None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:19.289073 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:19.297350 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 503/982] 10.222.0.89 () {60 vars in 1141 bytes} [Thu Jul 28 17:32:19 2022] POST /identity/v3/services => generated 242 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 17:32:19.306784 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196076b6-7264-4b66-b745-16aee0ebe7d9 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:19.354806 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c1c6b6-a512-4617-9e61-43028d71f9e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:19.355665 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c1c6b6-a512-4617-9e61-43028d71f9e8 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:19.356428 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c1c6b6-a512-4617-9e61-43028d71f9e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:19.356707 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c1c6b6-a512-4617-9e61-43028d71f9e8 None admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:19.357832 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c1c6b6-a512-4617-9e61-43028d71f9e8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:32:19.358155 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c1c6b6-a512-4617-9e61-43028d71f9e8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:19.359640 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c1c6b6-a512-4617-9e61-43028d71f9e8 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:19.379791 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 480/983] 10.222.0.89 () {56 vars in 1149 bytes} [Thu Jul 28 17:32:19 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:32:19.386385 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7e41af-9493-463b-88bc-0038f8c6978b None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:19.502290 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7e41af-9493-463b-88bc-0038f8c6978b None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:19.521177 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:19.540590 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22d51d7-386f-45cc-9f5f-baa75f8d7d90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:19.541134 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22d51d7-386f-45cc-9f5f-baa75f8d7d90 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:19.541444 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22d51d7-386f-45cc-9f5f-baa75f8d7d90 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:19.542181 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22d51d7-386f-45cc-9f5f-baa75f8d7d90 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:19.613472 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22d51d7-386f-45cc-9f5f-baa75f8d7d90 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:19.614928 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22d51d7-386f-45cc-9f5f-baa75f8d7d90 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:19.714961 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 504/984] 10.222.0.89 () {58 vars in 1282 bytes} [Thu Jul 28 17:32:19 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 17:32:19.720505 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c1c6b6-a512-4617-9e61-43028d71f9e8 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:19.831530 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c1c6b6-a512-4617-9e61-43028d71f9e8 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:19.866494 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3502f66b-3b81-4732-8583-357ac04c6843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:19.867142 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3502f66b-3b81-4732-8583-357ac04c6843 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:19.867420 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3502f66b-3b81-4732-8583-357ac04c6843 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:19.867691 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3502f66b-3b81-4732-8583-357ac04c6843 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:19.877272 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:19.880204 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3502f66b-3b81-4732-8583-357ac04c6843 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:19.881906 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3502f66b-3b81-4732-8583-357ac04c6843 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:19.899563 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-3502f66b-3b81-4732-8583-357ac04c6843 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:32:19.899563 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:32:19.899563 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:32:19.899563 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:32:19.899563 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:32:19.899563 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:32:19.899563 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:32:19.899563 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:32:19.899563 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:32:19.899563 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:32:19.899563 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:32:19.899563 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:32:19.899563 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:32:19.899563 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:32:19.899563 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:32:19.900573 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:32:19.900573 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:32:19.900573 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:32:19.900573 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:32:19.900573 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:32:19.900573 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:32:19.900573 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:32:19.900573 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:32:19.900573 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:32:19.900573 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:32:19.900573 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:32:19.900573 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:32:19.900573 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:32:19.900573 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:32:19.900573 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:32:19.900573 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:32:19.900573 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:32:19.901925 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:32:19.901925 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:32:19.901925 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:32:19.901925 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:32:19.901925 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 481/985] 10.222.0.89 () {56 vars in 1108 bytes} [Thu Jul 28 17:32:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:32:19.906386 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22d51d7-386f-45cc-9f5f-baa75f8d7d90 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:19.926015 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:20.014416 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22d51d7-386f-45cc-9f5f-baa75f8d7d90 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:20.048993 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08c29d9-66a6-49e5-9c1f-fbf5846d06fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:20.049638 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08c29d9-66a6-49e5-9c1f-fbf5846d06fe None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:20.049922 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08c29d9-66a6-49e5-9c1f-fbf5846d06fe None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:20.050193 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08c29d9-66a6-49e5-9c1f-fbf5846d06fe None admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:20.050937 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08c29d9-66a6-49e5-9c1f-fbf5846d06fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:32:20.051242 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08c29d9-66a6-49e5-9c1f-fbf5846d06fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:20.052602 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08c29d9-66a6-49e5-9c1f-fbf5846d06fe None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:20.069063 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 505/986] 10.222.0.89 () {56 vars in 1109 bytes} [Thu Jul 28 17:32:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:32:20.070429 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3502f66b-3b81-4732-8583-357ac04c6843 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:20.170931 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3502f66b-3b81-4732-8583-357ac04c6843 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:20.204455 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a323683-f80d-46db-8c84-36a6a19e4618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:20.205274 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a323683-f80d-46db-8c84-36a6a19e4618 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:20.205274 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a323683-f80d-46db-8c84-36a6a19e4618 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:20.205549 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a323683-f80d-46db-8c84-36a6a19e4618 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:20.212547 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a323683-f80d-46db-8c84-36a6a19e4618 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:20.214010 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a323683-f80d-46db-8c84-36a6a19e4618 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:20.225388 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-9a323683-f80d-46db-8c84-36a6a19e4618 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:32:20.225388 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:32:20.225388 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:32:20.225388 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:32:20.225388 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:32:20.225388 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:32:20.225388 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:32:20.225388 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:32:20.225388 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:32:20.225388 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:32:20.225388 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:32:20.225388 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:32:20.225388 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 17:32:20.225388 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 17:32:20.225388 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 17:32:20.226254 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 17:32:20.226254 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:32:20.226254 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:32:20.226254 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:32:20.226254 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:32:20.226254 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:32:20.226254 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:32:20.226254 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:32:20.226254 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:32:20.226254 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:32:20.226254 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:32:20.226254 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:32:20.226254 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:32:20.226254 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:32:20.226254 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:32:20.226254 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:32:20.226254 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:32:20.227153 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 17:32:20.227153 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 17:32:20.227153 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 17:32:20.227153 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 17:32:20.227153 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 17:32:20.227153 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 17:32:20.227153 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:32:20.227153 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:32:20.228125 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 482/987] 10.222.0.89 () {56 vars in 1111 bytes} [Thu Jul 28 17:32:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:32:20.232172 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08c29d9-66a6-49e5-9c1f-fbf5846d06fe None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:20.315847 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:20.339485 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08c29d9-66a6-49e5-9c1f-fbf5846d06fe None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:20.377295 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b33520-4549-4249-b08a-700db0db81d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:20.377295 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b33520-4549-4249-b08a-700db0db81d6 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:20.377295 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b33520-4549-4249-b08a-700db0db81d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:20.377295 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b33520-4549-4249-b08a-700db0db81d6 None admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:20.377995 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b33520-4549-4249-b08a-700db0db81d6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:32:20.377995 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b33520-4549-4249-b08a-700db0db81d6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:20.379031 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b33520-4549-4249-b08a-700db0db81d6 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:20.397007 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 506/988] 10.222.0.89 () {56 vars in 1148 bytes} [Thu Jul 28 17:32:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:32:20.398002 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a323683-f80d-46db-8c84-36a6a19e4618 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:20.452715 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:20.505460 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a323683-f80d-46db-8c84-36a6a19e4618 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:20.541407 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3168e87f-7ba7-46f8-bbd8-4878c01f1b3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:20.542021 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3168e87f-7ba7-46f8-bbd8-4878c01f1b3f None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:20.542295 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3168e87f-7ba7-46f8-bbd8-4878c01f1b3f None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:20.542564 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3168e87f-7ba7-46f8-bbd8-4878c01f1b3f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:20.543343 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3168e87f-7ba7-46f8-bbd8-4878c01f1b3f None admin] RBAC: Adding query filter params (role.id=af4b58a9c88d4cb4861277524ab3e1ed, scope.project.id=04434e8535a64044996316f1d17dc0bc, user.id=c44387d13a694ee6bb47a13906003330) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:32:20.543643 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3168e87f-7ba7-46f8-bbd8-4878c01f1b3f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:20.546260 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3168e87f-7ba7-46f8-bbd8-4878c01f1b3f None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:20.552389 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 483/989] 10.222.0.89 () {56 vars in 1374 bytes} [Thu Jul 28 17:32:20 2022] GET /identity/v3/role_assignments?user.id=c44387d13a694ee6bb47a13906003330&scope.project.id=04434e8535a64044996316f1d17dc0bc&role.id=af4b58a9c88d4cb4861277524ab3e1ed => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:32:20.879976 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcdb22c-7012-49f3-81ab-ecbf064dbd71 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:20.880493 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcdb22c-7012-49f3-81ab-ecbf064dbd71 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:20.880600 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcdb22c-7012-49f3-81ab-ecbf064dbd71 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:20.881926 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 507/990] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:32:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:32:20.887056 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ee12a7-f75a-44cb-ac67-f10d66baafaf None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:20.887349 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ee12a7-f75a-44cb-ac67-f10d66baafaf None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:20.887622 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ee12a7-f75a-44cb-ac67-f10d66baafaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:20.906174 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-29ee12a7-f75a-44cb-ac67-f10d66baafaf None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:32:20.907383 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-29ee12a7-f75a-44cb-ac67-f10d66baafaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:32:20.976044 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 484/991] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:32:20 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:32:20.999827 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcdb22c-7012-49f3-81ab-ecbf064dbd71 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:21.105716 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcdb22c-7012-49f3-81ab-ecbf064dbd71 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:21.140961 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b31edc-bfcf-4326-935b-0c3c2998f3e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:21.141629 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b31edc-bfcf-4326-935b-0c3c2998f3e5 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:21.141913 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b31edc-bfcf-4326-935b-0c3c2998f3e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:21.142187 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b31edc-bfcf-4326-935b-0c3c2998f3e5 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:21.142814 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b31edc-bfcf-4326-935b-0c3c2998f3e5 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:21.144196 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b31edc-bfcf-4326-935b-0c3c2998f3e5 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:21.152958 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-b5b31edc-bfcf-4326-935b-0c3c2998f3e5 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 17:32:21.152958 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:32:21.152958 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 17:32:21.152958 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 17:32:21.152958 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 17:32:21.152958 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 17:32:21.152958 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 17:32:21.152958 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:32:21.152958 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 17:32:21.152958 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:32:21.152958 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 17:32:21.152958 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:32:21.152958 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:32:21.152958 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:32:21.152958 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:32:21.152958 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:32:21.152958 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:32:21.153972 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:32:21.153972 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:32:21.153972 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:32:21.153972 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:32:21.153972 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:32:21.153972 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:32:21.153972 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 17:32:21.153972 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 17:32:21.153972 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 17:32:21.153972 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 17:32:21.153972 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:32:21.153972 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:32:21.153972 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:32:21.153972 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:32:21.153972 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:32:21.153972 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:32:21.155369 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:32:21.155369 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( Jul 28 17:32:21.155369 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:32:21.155369 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:32:21.155369 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:32:21.155369 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:32:21.155369 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:32:21.155369 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:32:21.155369 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:32:21.155369 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:32:21.155369 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 17:32:21.155369 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:32:21.155369 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 17:32:21.155369 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:32:21.156852 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 508/992] 10.222.0.89 () {56 vars in 1126 bytes} [Thu Jul 28 17:32:20 2022] GET /identity/v3/services/object-store => generated 93 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:32:21.161009 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ee12a7-f75a-44cb-ac67-f10d66baafaf None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:21.209226 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:21.263467 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ee12a7-f75a-44cb-ac67-f10d66baafaf None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:21.298502 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc02bb5f-d8a4-47f9-b9f2-639e13fcb1b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:21.299135 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc02bb5f-d8a4-47f9-b9f2-639e13fcb1b0 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:21.299417 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc02bb5f-d8a4-47f9-b9f2-639e13fcb1b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:21.299692 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc02bb5f-d8a4-47f9-b9f2-639e13fcb1b0 None admin] PATH_INFO: `/v3/services` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:21.300379 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc02bb5f-d8a4-47f9-b9f2-639e13fcb1b0 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:32:21.300686 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc02bb5f-d8a4-47f9-b9f2-639e13fcb1b0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:21.302087 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc02bb5f-d8a4-47f9-b9f2-639e13fcb1b0 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:21.312216 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 485/993] 10.222.0.89 () {56 vars in 1122 bytes} [Thu Jul 28 17:32:21 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:32:21.313504 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b31edc-bfcf-4326-935b-0c3c2998f3e5 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:21.316452 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:21.417918 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b31edc-bfcf-4326-935b-0c3c2998f3e5 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:21.436399 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95e9f5d-69ba-408d-9cfc-db5a9e1f6b36 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:21.436913 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95e9f5d-69ba-408d-9cfc-db5a9e1f6b36 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:21.437410 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95e9f5d-69ba-408d-9cfc-db5a9e1f6b36 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:21.438813 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 486/994] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:32:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 17:32:21.443857 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c41c1e-b751-4275-a524-dbc19df5e3a4 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:21.444142 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c41c1e-b751-4275-a524-dbc19df5e3a4 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:21.444419 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c41c1e-b751-4275-a524-dbc19df5e3a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:21.458305 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c0033a-b908-4514-adf9-08bda1291479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:21.458920 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c0033a-b908-4514-adf9-08bda1291479 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:21.459200 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c0033a-b908-4514-adf9-08bda1291479 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:21.459473 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c0033a-b908-4514-adf9-08bda1291479 None admin] PATH_INFO: `/v3/services` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:21.460148 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c0033a-b908-4514-adf9-08bda1291479 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:32:21.460452 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c0033a-b908-4514-adf9-08bda1291479 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:21.461843 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c0033a-b908-4514-adf9-08bda1291479 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:21.464697 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-02c41c1e-b751-4275-a524-dbc19df5e3a4 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:32:21.466186 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-02c41c1e-b751-4275-a524-dbc19df5e3a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:32:21.466873 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 509/995] 10.222.0.89 () {56 vars in 1122 bytes} [Thu Jul 28 17:32:21 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:32:21.472403 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c0033a-b908-4514-adf9-08bda1291479 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:21.539555 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:21.542391 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 487/996] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:32:21 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:32:21.565804 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c41c1e-b751-4275-a524-dbc19df5e3a4 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:21.577693 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c0033a-b908-4514-adf9-08bda1291479 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:21.614645 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6b68e1-284d-4d0d-a60d-41ab3257109b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:21.615266 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6b68e1-284d-4d0d-a60d-41ab3257109b None admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:21.615548 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6b68e1-284d-4d0d-a60d-41ab3257109b None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:21.615823 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6b68e1-284d-4d0d-a60d-41ab3257109b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:21.616537 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6b68e1-284d-4d0d-a60d-41ab3257109b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:21.618399 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6b68e1-284d-4d0d-a60d-41ab3257109b None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:21.619295 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:21.653221 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 510/997] 10.222.0.89 () {60 vars in 1144 bytes} [Thu Jul 28 17:32:21 2022] POST /identity/v3/endpoints => generated 325 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 17:32:21.691230 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c41c1e-b751-4275-a524-dbc19df5e3a4 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:21.739613 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d49172-0f88-49c2-9659-0259c2d943cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:21.740400 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d49172-0f88-49c2-9659-0259c2d943cd None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:21.740583 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d49172-0f88-49c2-9659-0259c2d943cd None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:21.740878 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d49172-0f88-49c2-9659-0259c2d943cd None admin] PATH_INFO: `/v3/services/compute` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:21.741548 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d49172-0f88-49c2-9659-0259c2d943cd None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:21.742984 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d49172-0f88-49c2-9659-0259c2d943cd None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:21.753786 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-09d49172-0f88-49c2-9659-0259c2d943cd None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 17:32:21.753786 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:32:21.753786 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 17:32:21.753786 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 17:32:21.753786 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 17:32:21.753786 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 17:32:21.753786 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 17:32:21.753786 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:32:21.753786 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 17:32:21.753786 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:32:21.753786 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 17:32:21.753786 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:32:21.753786 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:32:21.753786 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:32:21.753786 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:32:21.753786 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:32:21.753786 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:32:21.754931 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:32:21.754931 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:32:21.754931 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:32:21.754931 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:32:21.754931 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:32:21.754931 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:32:21.754931 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 17:32:21.754931 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 17:32:21.754931 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 17:32:21.754931 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 17:32:21.754931 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:32:21.754931 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:32:21.754931 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:32:21.754931 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:32:21.754931 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:32:21.754931 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:32:21.755822 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:32:21.755822 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:32:21.755822 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:32:21.755822 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:32:21.755822 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:32:21.755822 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:32:21.755822 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:32:21.755822 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:32:21.755822 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:32:21.755822 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:32:21.755822 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 17:32:21.755822 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:32:21.755822 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 17:32:21.755822 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:32:21.760306 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 488/998] 10.222.0.89 () {56 vars in 1111 bytes} [Thu Jul 28 17:32:21 2022] GET /identity/v3/services/compute => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:32:21.761605 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6b68e1-284d-4d0d-a60d-41ab3257109b None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:21.876008 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6b68e1-284d-4d0d-a60d-41ab3257109b None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:21.903317 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:21.915475 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf68f37-9eac-4f62-94b4-f0720d6c19a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:21.916293 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf68f37-9eac-4f62-94b4-f0720d6c19a7 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:21.916749 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf68f37-9eac-4f62-94b4-f0720d6c19a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:21.917220 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf68f37-9eac-4f62-94b4-f0720d6c19a7 None admin] PATH_INFO: `/v3/services` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:21.918045 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf68f37-9eac-4f62-94b4-f0720d6c19a7 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:32:21.918498 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf68f37-9eac-4f62-94b4-f0720d6c19a7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:21.920278 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf68f37-9eac-4f62-94b4-f0720d6c19a7 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:21.925465 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 511/999] 10.222.0.89 () {56 vars in 1112 bytes} [Thu Jul 28 17:32:21 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:32:21.931308 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d49172-0f88-49c2-9659-0259c2d943cd None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:22.034746 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:22.038604 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d49172-0f88-49c2-9659-0259c2d943cd None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:22.073284 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ddf586-b892-4a72-89a5-33a880c7f24e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:22.073974 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ddf586-b892-4a72-89a5-33a880c7f24e None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:22.074266 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ddf586-b892-4a72-89a5-33a880c7f24e None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:22.074544 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ddf586-b892-4a72-89a5-33a880c7f24e None admin] PATH_INFO: `/v3/services` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:22.075225 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ddf586-b892-4a72-89a5-33a880c7f24e None admin] RBAC: Adding query filter params (type=compute) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:32:22.075532 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ddf586-b892-4a72-89a5-33a880c7f24e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:22.077707 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ddf586-b892-4a72-89a5-33a880c7f24e None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:22.087514 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 489/1000] 10.222.0.89 () {56 vars in 1112 bytes} [Thu Jul 28 17:32:21 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:32:22.089061 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf68f37-9eac-4f62-94b4-f0720d6c19a7 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:22.195621 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf68f37-9eac-4f62-94b4-f0720d6c19a7 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:22.229806 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab2bfc6-5269-4dd5-9506-a53629d8e39b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:22.230422 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab2bfc6-5269-4dd5-9506-a53629d8e39b None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:22.230706 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab2bfc6-5269-4dd5-9506-a53629d8e39b None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:22.230979 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab2bfc6-5269-4dd5-9506-a53629d8e39b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:22.231759 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab2bfc6-5269-4dd5-9506-a53629d8e39b None admin] RBAC: Adding query filter params (interface=public, service_id=885dfde2679c44cab1ec8524eff5d545, region_id=RegionOne) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:32:22.232063 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab2bfc6-5269-4dd5-9506-a53629d8e39b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:22.233496 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab2bfc6-5269-4dd5-9506-a53629d8e39b None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:22.243083 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 512/1001] 10.222.0.89 () {56 vars in 1251 bytes} [Thu Jul 28 17:32:22 2022] GET /identity/v3/endpoints?service_id=885dfde2679c44cab1ec8524eff5d545&interface=public®ion_id=RegionOne => generated 194 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:32:22.244671 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ddf586-b892-4a72-89a5-33a880c7f24e None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:22.350504 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ddf586-b892-4a72-89a5-33a880c7f24e None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:22.385897 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af1cc17-bf63-4898-9977-388d99378054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:22.386513 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af1cc17-bf63-4898-9977-388d99378054 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:22.386790 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af1cc17-bf63-4898-9977-388d99378054 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:22.387057 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af1cc17-bf63-4898-9977-388d99378054 None admin] PATH_INFO: `/v3/services` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:22.387702 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af1cc17-bf63-4898-9977-388d99378054 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:22.389718 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af1cc17-bf63-4898-9977-388d99378054 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:22.405186 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 490/1002] 10.222.0.89 () {56 vars in 1087 bytes} [Thu Jul 28 17:32:22 2022] GET /identity/v3/services => generated 1727 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 17:32:22.614478 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a48917a-2bd4-4ecb-9749-2bef700955d7 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:22.614849 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a48917a-2bd4-4ecb-9749-2bef700955d7 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:22.614968 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a48917a-2bd4-4ecb-9749-2bef700955d7 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:22.616429 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 513/1003] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:32:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:32:22.621904 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b39eb8-bb47-48a7-95e7-dca4c43310c4 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:22.622369 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b39eb8-bb47-48a7-95e7-dca4c43310c4 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:22.622449 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b39eb8-bb47-48a7-95e7-dca4c43310c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:22.640944 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-76b39eb8-bb47-48a7-95e7-dca4c43310c4 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:32:22.642173 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-76b39eb8-bb47-48a7-95e7-dca4c43310c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:32:22.665921 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:22.711200 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 491/1004] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:32:22 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:32:22.734075 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a48917a-2bd4-4ecb-9749-2bef700955d7 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:22.794366 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:22.852684 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a48917a-2bd4-4ecb-9749-2bef700955d7 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:22.891840 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5f788c-0219-4fdc-bbb1-ce9b2a1a7f51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:22.892845 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5f788c-0219-4fdc-bbb1-ce9b2a1a7f51 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:22.893366 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5f788c-0219-4fdc-bbb1-ce9b2a1a7f51 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:22.893847 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5f788c-0219-4fdc-bbb1-ce9b2a1a7f51 None admin] PATH_INFO: `/v3/services/image` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:22.894675 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5f788c-0219-4fdc-bbb1-ce9b2a1a7f51 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:22.896245 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5f788c-0219-4fdc-bbb1-ce9b2a1a7f51 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:22.906175 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-ab5f788c-0219-4fdc-bbb1-ce9b2a1a7f51 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 17:32:22.906175 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:32:22.906175 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 17:32:22.906175 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 17:32:22.906175 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 17:32:22.906175 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 17:32:22.906175 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 17:32:22.906175 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:32:22.906175 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 17:32:22.906175 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:32:22.906175 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 17:32:22.906175 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:32:22.906175 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:32:22.906175 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:32:22.906175 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:32:22.906175 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:32:22.906175 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:32:22.907629 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:32:22.907629 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:32:22.907629 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:32:22.907629 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:32:22.907629 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:32:22.907629 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:32:22.907629 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 17:32:22.907629 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 17:32:22.907629 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 17:32:22.907629 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 17:32:22.907629 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:32:22.907629 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:32:22.907629 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:32:22.907629 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:32:22.907629 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:32:22.907629 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:32:22.908513 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:32:22.908513 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( Jul 28 17:32:22.908513 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:32:22.908513 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:32:22.908513 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:32:22.908513 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:32:22.908513 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:32:22.908513 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:32:22.908513 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:32:22.908513 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:32:22.908513 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 17:32:22.908513 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:32:22.908513 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 17:32:22.908513 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:32:22.909658 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 514/1005] 10.222.0.89 () {56 vars in 1105 bytes} [Thu Jul 28 17:32:22 2022] GET /identity/v3/services/image => generated 86 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:32:22.914949 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b39eb8-bb47-48a7-95e7-dca4c43310c4 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:23.017965 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b39eb8-bb47-48a7-95e7-dca4c43310c4 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:23.054985 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44c9058-b994-4e77-b756-aaeec489bcfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:23.055614 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44c9058-b994-4e77-b756-aaeec489bcfe None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:23.055898 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44c9058-b994-4e77-b756-aaeec489bcfe None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:23.056174 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44c9058-b994-4e77-b756-aaeec489bcfe None admin] PATH_INFO: `/v3/services` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:23.056871 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44c9058-b994-4e77-b756-aaeec489bcfe None admin] RBAC: Adding query filter params (name=image) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:32:23.057206 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44c9058-b994-4e77-b756-aaeec489bcfe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:23.058583 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44c9058-b994-4e77-b756-aaeec489bcfe None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:23.068911 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 492/1006] 10.222.0.89 () {56 vars in 1108 bytes} [Thu Jul 28 17:32:22 2022] GET /identity/v3/services?name=image => generated 122 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:32:23.070310 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5f788c-0219-4fdc-bbb1-ce9b2a1a7f51 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:23.176345 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5f788c-0219-4fdc-bbb1-ce9b2a1a7f51 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:23.194877 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:23.213410 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53292b4f-e87c-4ecf-8e8f-fa421c47390c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:23.214026 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53292b4f-e87c-4ecf-8e8f-fa421c47390c None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:23.214308 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53292b4f-e87c-4ecf-8e8f-fa421c47390c None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:23.214584 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53292b4f-e87c-4ecf-8e8f-fa421c47390c None admin] PATH_INFO: `/v3/services` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:23.215263 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53292b4f-e87c-4ecf-8e8f-fa421c47390c None admin] RBAC: Adding query filter params (type=image) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:32:23.215568 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53292b4f-e87c-4ecf-8e8f-fa421c47390c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:23.217022 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53292b4f-e87c-4ecf-8e8f-fa421c47390c None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:23.229209 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 515/1007] 10.222.0.89 () {56 vars in 1108 bytes} [Thu Jul 28 17:32:23 2022] GET /identity/v3/services?type=image => generated 122 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:32:23.710126 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c31e70-3f24-4e30-bb74-7b3cff4622bc None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:23.710653 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c31e70-3f24-4e30-bb74-7b3cff4622bc None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:23.710756 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c31e70-3f24-4e30-bb74-7b3cff4622bc None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:23.712086 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 493/1008] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:32:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:32:23.717788 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10cc5cf-6444-4a12-a0cf-eaf3589a98ce None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:23.718076 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10cc5cf-6444-4a12-a0cf-eaf3589a98ce None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:23.718353 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10cc5cf-6444-4a12-a0cf-eaf3589a98ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:23.736960 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-f10cc5cf-6444-4a12-a0cf-eaf3589a98ce None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:32:23.738204 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-f10cc5cf-6444-4a12-a0cf-eaf3589a98ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:32:23.806101 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 516/1009] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:32:23 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:32:23.830035 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c31e70-3f24-4e30-bb74-7b3cff4622bc None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:23.855381 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:23.936232 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c31e70-3f24-4e30-bb74-7b3cff4622bc None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:23.972471 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ecf33a-08de-4b05-b32e-4f1eb6a5f5f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:23.973037 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ecf33a-08de-4b05-b32e-4f1eb6a5f5f8 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:23.973347 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ecf33a-08de-4b05-b32e-4f1eb6a5f5f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:23.973655 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ecf33a-08de-4b05-b32e-4f1eb6a5f5f8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:23.974783 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ecf33a-08de-4b05-b32e-4f1eb6a5f5f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:23.976223 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ecf33a-08de-4b05-b32e-4f1eb6a5f5f8 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:23.984171 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 494/1010] 10.222.0.89 () {56 vars in 1108 bytes} [Thu Jul 28 17:32:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:32:23.985128 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10cc5cf-6444-4a12-a0cf-eaf3589a98ce None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:24.086291 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10cc5cf-6444-4a12-a0cf-eaf3589a98ce None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:24.099543 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:24.123564 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce569b9a-7f1f-4a25-8182-20d58801c21d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:24.124430 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce569b9a-7f1f-4a25-8182-20d58801c21d None admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:24.124837 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce569b9a-7f1f-4a25-8182-20d58801c21d None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:24.124961 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce569b9a-7f1f-4a25-8182-20d58801c21d None admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:24.125642 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce569b9a-7f1f-4a25-8182-20d58801c21d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:24.127068 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce569b9a-7f1f-4a25-8182-20d58801c21d None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:24.155689 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 517/1011] 10.222.0.89 () {60 vars in 1141 bytes} [Thu Jul 28 17:32:23 2022] POST /identity/v3/projects => generated 311 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 17:32:24.450504 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e4d1b3-2f32-4572-8faf-4dadc589c71c None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:24.450872 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e4d1b3-2f32-4572-8faf-4dadc589c71c None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:24.451151 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e4d1b3-2f32-4572-8faf-4dadc589c71c None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:24.453183 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 495/1012] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:32:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:32:24.458216 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8045d8-c6cd-4b5f-9bab-80abda246baf None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:24.458511 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8045d8-c6cd-4b5f-9bab-80abda246baf None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:24.458787 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8045d8-c6cd-4b5f-9bab-80abda246baf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:24.477229 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-ac8045d8-c6cd-4b5f-9bab-80abda246baf None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:32:24.478427 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-ac8045d8-c6cd-4b5f-9bab-80abda246baf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:32:24.548737 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 518/1013] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:32:24 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:32:24.570969 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e4d1b3-2f32-4572-8faf-4dadc589c71c None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:24.674336 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:24.685774 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e4d1b3-2f32-4572-8faf-4dadc589c71c None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:24.721798 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4a4c18-73d1-40c1-8832-4d8a4b5fd143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:24.722432 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4a4c18-73d1-40c1-8832-4d8a4b5fd143 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:24.722658 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4a4c18-73d1-40c1-8832-4d8a4b5fd143 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:24.722901 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4a4c18-73d1-40c1-8832-4d8a4b5fd143 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:24.723533 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4a4c18-73d1-40c1-8832-4d8a4b5fd143 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:24.724898 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4a4c18-73d1-40c1-8832-4d8a4b5fd143 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:24.734147 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-cf4a4c18-73d1-40c1-8832-4d8a4b5fd143 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 17:32:24.734147 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:32:24.734147 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 17:32:24.734147 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 17:32:24.734147 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 17:32:24.734147 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 17:32:24.734147 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 17:32:24.734147 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:32:24.734147 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 17:32:24.734147 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:32:24.734147 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 17:32:24.734147 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:32:24.734147 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:32:24.734147 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:32:24.734147 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:32:24.734147 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:32:24.734147 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:32:24.735178 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:32:24.735178 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:32:24.735178 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:32:24.735178 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:32:24.735178 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:32:24.735178 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:32:24.735178 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 17:32:24.735178 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 17:32:24.735178 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 17:32:24.735178 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 17:32:24.735178 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:32:24.735178 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:32:24.735178 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:32:24.735178 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:32:24.735178 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:32:24.735178 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:32:24.736031 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:32:24.736031 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:32:24.736031 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:32:24.736031 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:32:24.736031 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:32:24.736031 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:32:24.736031 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:32:24.736031 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:32:24.736031 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:32:24.736031 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:32:24.736031 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 17:32:24.736031 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:32:24.736031 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 17:32:24.736031 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:32:24.736031 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 496/1014] 10.222.0.89 () {56 vars in 1111 bytes} [Thu Jul 28 17:32:24 2022] GET /identity/v3/services/compute => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:32:24.743033 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8045d8-c6cd-4b5f-9bab-80abda246baf None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:24.846154 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:24.854542 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8045d8-c6cd-4b5f-9bab-80abda246baf None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:24.892022 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027b30ee-8dbc-4970-adba-cb78d7e9ad25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:24.892642 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027b30ee-8dbc-4970-adba-cb78d7e9ad25 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:24.892922 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027b30ee-8dbc-4970-adba-cb78d7e9ad25 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:24.893219 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027b30ee-8dbc-4970-adba-cb78d7e9ad25 None admin] PATH_INFO: `/v3/services` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:24.893899 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027b30ee-8dbc-4970-adba-cb78d7e9ad25 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:32:24.894202 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027b30ee-8dbc-4970-adba-cb78d7e9ad25 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:24.895625 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027b30ee-8dbc-4970-adba-cb78d7e9ad25 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:24.906170 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 519/1015] 10.222.0.89 () {56 vars in 1112 bytes} [Thu Jul 28 17:32:24 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:32:24.907468 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4a4c18-73d1-40c1-8832-4d8a4b5fd143 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:25.011544 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4a4c18-73d1-40c1-8832-4d8a4b5fd143 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:25.045623 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc8595b-98d7-4d87-b5d2-8da7904ba2ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:25.046147 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc8595b-98d7-4d87-b5d2-8da7904ba2ae None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:25.046429 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc8595b-98d7-4d87-b5d2-8da7904ba2ae None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:25.046711 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc8595b-98d7-4d87-b5d2-8da7904ba2ae None admin] PATH_INFO: `/v3/services` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:25.047392 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc8595b-98d7-4d87-b5d2-8da7904ba2ae None admin] RBAC: Adding query filter params (type=compute) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:32:25.047698 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc8595b-98d7-4d87-b5d2-8da7904ba2ae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:25.049056 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc8595b-98d7-4d87-b5d2-8da7904ba2ae None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:25.058603 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 497/1016] 10.222.0.89 () {56 vars in 1112 bytes} [Thu Jul 28 17:32:24 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:32:25.059812 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027b30ee-8dbc-4970-adba-cb78d7e9ad25 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:25.164504 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027b30ee-8dbc-4970-adba-cb78d7e9ad25 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:25.199274 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d338a5-072f-4bb3-95d4-2bb8386a0153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:25.199892 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d338a5-072f-4bb3-95d4-2bb8386a0153 None admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:25.200172 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d338a5-072f-4bb3-95d4-2bb8386a0153 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:25.200442 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d338a5-072f-4bb3-95d4-2bb8386a0153 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:25.201834 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d338a5-072f-4bb3-95d4-2bb8386a0153 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:25.203247 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d338a5-072f-4bb3-95d4-2bb8386a0153 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:25.227017 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 520/1017] 10.222.0.89 () {60 vars in 1144 bytes} [Thu Jul 28 17:32:25 2022] POST /identity/v3/endpoints => generated 334 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:32:25.243642 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fcfe3e-5b2c-48cb-b8ae-9cbe11b4ed30 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:25.243956 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fcfe3e-5b2c-48cb-b8ae-9cbe11b4ed30 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:25.244205 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fcfe3e-5b2c-48cb-b8ae-9cbe11b4ed30 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:25.245832 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 498/1018] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:32:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:32:25.252510 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab42c0e9-d479-4361-bdc4-b8fbca64aa95 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:25.252820 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab42c0e9-d479-4361-bdc4-b8fbca64aa95 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:25.253077 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab42c0e9-d479-4361-bdc4-b8fbca64aa95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:25.273513 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-ab42c0e9-d479-4361-bdc4-b8fbca64aa95 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:32:25.274652 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-ab42c0e9-d479-4361-bdc4-b8fbca64aa95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:32:25.286085 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:25.364099 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 521/1019] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:32:25 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:32:25.386840 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fcfe3e-5b2c-48cb-b8ae-9cbe11b4ed30 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:25.409520 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:25.496488 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fcfe3e-5b2c-48cb-b8ae-9cbe11b4ed30 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:25.534930 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e608c7b-96cc-41f6-a3ca-85ad3b3e4c7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:25.535471 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e608c7b-96cc-41f6-a3ca-85ad3b3e4c7a None admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:25.535750 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e608c7b-96cc-41f6-a3ca-85ad3b3e4c7a None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:25.536022 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e608c7b-96cc-41f6-a3ca-85ad3b3e4c7a None admin] PATH_INFO: `/v3/services` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:25.536729 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e608c7b-96cc-41f6-a3ca-85ad3b3e4c7a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:25.538118 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e608c7b-96cc-41f6-a3ca-85ad3b3e4c7a None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:25.557216 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 499/1020] 10.222.0.89 () {60 vars in 1141 bytes} [Thu Jul 28 17:32:25 2022] POST /identity/v3/services => generated 242 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:32:26.284151 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b80a2f-8358-4c17-8793-3c2b64d7b30a None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:26.284713 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b80a2f-8358-4c17-8793-3c2b64d7b30a None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:26.284713 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b80a2f-8358-4c17-8793-3c2b64d7b30a None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:26.286282 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 522/1021] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:32:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:32:26.291660 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9606f9e2-cb5b-4ebc-8e6e-aa2593922ba5 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:26.292130 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9606f9e2-cb5b-4ebc-8e6e-aa2593922ba5 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:26.292222 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9606f9e2-cb5b-4ebc-8e6e-aa2593922ba5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:26.312485 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-9606f9e2-cb5b-4ebc-8e6e-aa2593922ba5 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:32:26.313717 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-9606f9e2-cb5b-4ebc-8e6e-aa2593922ba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:32:26.393193 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 500/1022] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:32:26 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:32:26.414281 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b80a2f-8358-4c17-8793-3c2b64d7b30a None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:26.523931 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b80a2f-8358-4c17-8793-3c2b64d7b30a None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:26.560193 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c46f026-cee2-4c25-8e4d-dadb43d59687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:26.561703 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c46f026-cee2-4c25-8e4d-dadb43d59687 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:26.562219 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c46f026-cee2-4c25-8e4d-dadb43d59687 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:26.562679 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c46f026-cee2-4c25-8e4d-dadb43d59687 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:26.564131 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c46f026-cee2-4c25-8e4d-dadb43d59687 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:26.565802 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c46f026-cee2-4c25-8e4d-dadb43d59687 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:26.567881 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 523/1023] 10.222.0.89 () {56 vars in 1108 bytes} [Thu Jul 28 17:32:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:32:26.573455 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9606f9e2-cb5b-4ebc-8e6e-aa2593922ba5 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:26.578667 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:26.679593 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9606f9e2-cb5b-4ebc-8e6e-aa2593922ba5 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:26.713378 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420713fa-9c4f-4b8b-a9ff-16e0d42aafd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:26.714040 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420713fa-9c4f-4b8b-a9ff-16e0d42aafd5 None admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:26.714313 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420713fa-9c4f-4b8b-a9ff-16e0d42aafd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:26.714586 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420713fa-9c4f-4b8b-a9ff-16e0d42aafd5 None admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:26.715295 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420713fa-9c4f-4b8b-a9ff-16e0d42aafd5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:26.717676 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420713fa-9c4f-4b8b-a9ff-16e0d42aafd5 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:26.749186 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 501/1024] 10.222.0.89 () {60 vars in 1132 bytes} [Thu Jul 28 17:32:26 2022] POST /identity/v3/users => generated 285 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:32:27.391781 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50f8be7-68eb-4f48-a1b5-d3ce9e36e08b None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:27.392313 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50f8be7-68eb-4f48-a1b5-d3ce9e36e08b None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:27.392313 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50f8be7-68eb-4f48-a1b5-d3ce9e36e08b None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:27.393679 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 524/1025] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:32:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:32:27.399054 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2095d82-00b7-49be-9304-f983fd1e70a3 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:27.399497 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2095d82-00b7-49be-9304-f983fd1e70a3 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:27.399579 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2095d82-00b7-49be-9304-f983fd1e70a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:27.418611 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-d2095d82-00b7-49be-9304-f983fd1e70a3 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:32:27.420119 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-d2095d82-00b7-49be-9304-f983fd1e70a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:32:27.479372 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:27.494246 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 502/1026] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:32:27 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:32:27.519573 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50f8be7-68eb-4f48-a1b5-d3ce9e36e08b None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:27.530528 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:27.626764 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50f8be7-68eb-4f48-a1b5-d3ce9e36e08b None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:27.642806 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f15bfe3-e203-4d73-85c3-c7d414f4421e None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:27.643013 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f15bfe3-e203-4d73-85c3-c7d414f4421e None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:27.643295 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f15bfe3-e203-4d73-85c3-c7d414f4421e None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:27.644790 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 503/1027] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:32:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 17:32:27.649759 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095bf138-77a8-4421-aa3e-55326f7d0a39 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:27.650040 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095bf138-77a8-4421-aa3e-55326f7d0a39 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:27.650316 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095bf138-77a8-4421-aa3e-55326f7d0a39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:27.662952 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a6db89-43f1-43d4-ba75-9a196be9caaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:27.663821 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a6db89-43f1-43d4-ba75-9a196be9caaa None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:27.664275 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a6db89-43f1-43d4-ba75-9a196be9caaa None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:27.664708 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a6db89-43f1-43d4-ba75-9a196be9caaa None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:27.669286 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-095bf138-77a8-4421-aa3e-55326f7d0a39 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:32:27.671519 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-095bf138-77a8-4421-aa3e-55326f7d0a39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:32:27.679416 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a6db89-43f1-43d4-ba75-9a196be9caaa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:32:27.685627 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-84a6db89-43f1-43d4-ba75-9a196be9caaa None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 17:32:27.685627 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:32:27.685627 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:32:27.685627 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:32:27.685627 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:32:27.685627 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:32:27.685627 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:32:27.685627 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:32:27.685627 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:32:27.685627 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:32:27.685627 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:32:27.685627 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:32:27.685627 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:32:27.685627 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:32:27.685627 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:32:27.685627 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:32:27.686656 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:32:27.686656 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:32:27.686656 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:32:27.686656 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:32:27.686656 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:32:27.686656 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:32:27.686656 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:32:27.686656 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:32:27.686656 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:32:27.686656 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:32:27.686656 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:32:27.686656 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( Jul 28 17:32:27.686656 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:32:27.686656 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:32:27.686656 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:32:27.686656 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:32:27.688027 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:32:27.688027 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:32:27.688027 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:32:27.688027 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:32:27.688027 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:32:27.688027 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:32:27.688027 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:32:27.688027 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:32:27.688027 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:32:27.688027 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:32:27.688027 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 17:32:27.688027 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:32:27.688027 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 525/1028] 10.222.0.89 () {56 vars in 1120 bytes} [Thu Jul 28 17:32:27 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:32:27.693488 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a6db89-43f1-43d4-ba75-9a196be9caaa None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:27.753200 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 504/1029] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:32:27 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 17:32:27.771213 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:27.774226 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095bf138-77a8-4421-aa3e-55326f7d0a39 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:27.812436 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a6db89-43f1-43d4-ba75-9a196be9caaa None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:27.853791 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0477dba-e689-42c9-9f21-de15d323a9ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:27.854675 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0477dba-e689-42c9-9f21-de15d323a9ed None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:27.855127 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0477dba-e689-42c9-9f21-de15d323a9ed None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:27.855592 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0477dba-e689-42c9-9f21-de15d323a9ed None admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:27.856440 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0477dba-e689-42c9-9f21-de15d323a9ed None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:32:27.856901 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0477dba-e689-42c9-9f21-de15d323a9ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:27.858375 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0477dba-e689-42c9-9f21-de15d323a9ed None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:27.869327 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 526/1030] 10.222.0.89 () {56 vars in 1115 bytes} [Thu Jul 28 17:32:27 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:32:27.873451 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0477dba-e689-42c9-9f21-de15d323a9ed None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:27.891062 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095bf138-77a8-4421-aa3e-55326f7d0a39 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:27.912807 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:27.931674 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d35543-1d29-4c46-a1a5-2178ab12b18e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:27.932361 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d35543-1d29-4c46-a1a5-2178ab12b18e None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:27.932641 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d35543-1d29-4c46-a1a5-2178ab12b18e None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:27.932913 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d35543-1d29-4c46-a1a5-2178ab12b18e None admin] PATH_INFO: `/v3/services/image` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:27.934165 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d35543-1d29-4c46-a1a5-2178ab12b18e None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:27.936129 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d35543-1d29-4c46-a1a5-2178ab12b18e None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:27.945616 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-19d35543-1d29-4c46-a1a5-2178ab12b18e None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 17:32:27.945616 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:32:27.945616 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 17:32:27.945616 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 17:32:27.945616 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 17:32:27.945616 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 17:32:27.945616 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 17:32:27.945616 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:32:27.945616 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 17:32:27.945616 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:32:27.945616 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 17:32:27.945616 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:32:27.945616 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:32:27.945616 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:32:27.945616 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:32:27.945616 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:32:27.945616 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:32:27.946639 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:32:27.946639 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:32:27.946639 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:32:27.946639 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:32:27.946639 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:32:27.946639 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:32:27.946639 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 17:32:27.946639 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 17:32:27.946639 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 17:32:27.946639 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 17:32:27.946639 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:32:27.946639 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:32:27.946639 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:32:27.946639 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:32:27.946639 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:32:27.946639 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:32:27.947530 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:32:27.947530 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:32:27.947530 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:32:27.947530 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:32:27.947530 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:32:27.947530 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:32:27.947530 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:32:27.947530 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:32:27.947530 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:32:27.947530 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:32:27.947530 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 17:32:27.947530 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:32:27.947530 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 17:32:27.947530 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:32:27.952559 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 505/1031] 10.222.0.89 () {56 vars in 1105 bytes} [Thu Jul 28 17:32:27 2022] GET /identity/v3/services/image => generated 86 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:32:27.953262 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d35543-1d29-4c46-a1a5-2178ab12b18e None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:27.989885 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0477dba-e689-42c9-9f21-de15d323a9ed None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:28.021308 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:28.028223 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7156a1aa-8a9c-4e96-9822-dd449293be19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:28.029877 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7156a1aa-8a9c-4e96-9822-dd449293be19 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:28.030361 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7156a1aa-8a9c-4e96-9822-dd449293be19 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:28.030686 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7156a1aa-8a9c-4e96-9822-dd449293be19 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:28.042090 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7156a1aa-8a9c-4e96-9822-dd449293be19 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:28.043529 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7156a1aa-8a9c-4e96-9822-dd449293be19 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:28.058491 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d35543-1d29-4c46-a1a5-2178ab12b18e None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:28.059820 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-7156a1aa-8a9c-4e96-9822-dd449293be19 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:32:28.059820 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:32:28.059820 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:32:28.059820 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:32:28.059820 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:32:28.059820 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:32:28.059820 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:32:28.059820 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:32:28.059820 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:32:28.059820 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:32:28.059820 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:32:28.059820 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:32:28.059820 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:32:28.059820 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:32:28.059820 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:32:28.060741 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:32:28.060741 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:32:28.060741 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:32:28.060741 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:32:28.060741 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:32:28.060741 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:32:28.060741 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:32:28.060741 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:32:28.060741 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( Jul 28 17:32:28.060741 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:32:28.060741 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:32:28.060741 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:32:28.060741 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:32:28.060741 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:32:28.060741 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:32:28.060741 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:32:28.060741 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:32:28.061672 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:32:28.061672 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:32:28.061672 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:32:28.061672 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:32:28.066893 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 527/1032] 10.222.0.89 () {56 vars in 1108 bytes} [Thu Jul 28 17:32:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:32:28.067607 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7156a1aa-8a9c-4e96-9822-dd449293be19 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:28.099777 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6221e09a-41d4-4c33-a2b6-7f0a5a405aa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:28.100396 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6221e09a-41d4-4c33-a2b6-7f0a5a405aa0 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:28.100675 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6221e09a-41d4-4c33-a2b6-7f0a5a405aa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:28.101098 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6221e09a-41d4-4c33-a2b6-7f0a5a405aa0 None admin] PATH_INFO: `/v3/services` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:28.101653 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6221e09a-41d4-4c33-a2b6-7f0a5a405aa0 None admin] RBAC: Adding query filter params (name=image) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:32:28.101984 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6221e09a-41d4-4c33-a2b6-7f0a5a405aa0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:28.103309 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6221e09a-41d4-4c33-a2b6-7f0a5a405aa0 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:28.110225 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 506/1033] 10.222.0.89 () {56 vars in 1108 bytes} [Thu Jul 28 17:32:27 2022] GET /identity/v3/services?name=image => generated 122 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:32:28.115611 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6221e09a-41d4-4c33-a2b6-7f0a5a405aa0 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:28.168118 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:28.174626 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7156a1aa-8a9c-4e96-9822-dd449293be19 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:28.210974 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdb62b3-84f6-4de1-82a0-780303a5c511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:28.211617 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdb62b3-84f6-4de1-82a0-780303a5c511 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:28.212333 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdb62b3-84f6-4de1-82a0-780303a5c511 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:28.212618 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdb62b3-84f6-4de1-82a0-780303a5c511 None admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:28.213339 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdb62b3-84f6-4de1-82a0-780303a5c511 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:32:28.213643 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdb62b3-84f6-4de1-82a0-780303a5c511 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:28.215008 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdb62b3-84f6-4de1-82a0-780303a5c511 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:28.225180 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6221e09a-41d4-4c33-a2b6-7f0a5a405aa0 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:28.227414 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 528/1034] 10.222.0.89 () {56 vars in 1109 bytes} [Thu Jul 28 17:32:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:32:28.233781 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdb62b3-84f6-4de1-82a0-780303a5c511 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:28.261434 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efbae9b-6f0e-4666-88cc-90b178bdebcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:28.262333 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efbae9b-6f0e-4666-88cc-90b178bdebcf None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:28.262617 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efbae9b-6f0e-4666-88cc-90b178bdebcf None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:28.262892 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efbae9b-6f0e-4666-88cc-90b178bdebcf None admin] PATH_INFO: `/v3/services` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:28.264369 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efbae9b-6f0e-4666-88cc-90b178bdebcf None admin] RBAC: Adding query filter params (type=image) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:32:28.265183 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efbae9b-6f0e-4666-88cc-90b178bdebcf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:28.266980 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efbae9b-6f0e-4666-88cc-90b178bdebcf None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:28.278479 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 507/1035] 10.222.0.89 () {56 vars in 1108 bytes} [Thu Jul 28 17:32:28 2022] GET /identity/v3/services?type=image => generated 350 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:32:28.280294 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efbae9b-6f0e-4666-88cc-90b178bdebcf None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:28.280859 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:28.342906 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdb62b3-84f6-4de1-82a0-780303a5c511 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:28.382021 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209b9d19-6125-4aa6-98a4-ce1f079d46d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:28.382662 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209b9d19-6125-4aa6-98a4-ce1f079d46d7 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:28.382943 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209b9d19-6125-4aa6-98a4-ce1f079d46d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:28.383218 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209b9d19-6125-4aa6-98a4-ce1f079d46d7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:28.391591 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209b9d19-6125-4aa6-98a4-ce1f079d46d7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:28.393034 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209b9d19-6125-4aa6-98a4-ce1f079d46d7 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:28.396679 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efbae9b-6f0e-4666-88cc-90b178bdebcf None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:28.405691 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-209b9d19-6125-4aa6-98a4-ce1f079d46d7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 17:32:28.405691 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:32:28.405691 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:32:28.405691 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:32:28.405691 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:32:28.405691 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:32:28.405691 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:32:28.405691 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:32:28.405691 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:32:28.405691 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:32:28.405691 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:32:28.405691 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:32:28.405691 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 17:32:28.405691 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 17:32:28.405691 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 17:32:28.405691 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 17:32:28.406669 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:32:28.406669 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:32:28.406669 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 17:32:28.406669 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:32:28.406669 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 17:32:28.406669 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:32:28.406669 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:32:28.406669 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:32:28.406669 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:32:28.406669 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:32:28.406669 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:32:28.406669 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( Jul 28 17:32:28.406669 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:32:28.406669 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:32:28.406669 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:32:28.406669 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:32:28.406669 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:32:28.406669 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:32:28.412535 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:32:28.412535 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:32:28.412535 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 17:32:28.412535 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 17:32:28.412535 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 17:32:28.412535 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 17:32:28.412535 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 17:32:28.412535 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 17:32:28.412535 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 17:32:28.412535 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:32:28.412535 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 529/1036] 10.222.0.89 () {56 vars in 1093 bytes} [Thu Jul 28 17:32:28 2022] GET /identity/v3/users/nova => generated 82 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:32:28.414368 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209b9d19-6125-4aa6-98a4-ce1f079d46d7 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:28.419220 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:28.445685 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a551ba-da2b-4837-a2fa-1af048121f34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:28.446301 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a551ba-da2b-4837-a2fa-1af048121f34 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:28.446593 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a551ba-da2b-4837-a2fa-1af048121f34 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:28.446846 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a551ba-da2b-4837-a2fa-1af048121f34 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:28.447596 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a551ba-da2b-4837-a2fa-1af048121f34 None admin] RBAC: Adding query filter params (interface=public, service_id=bda3f4af2ff6410db475d094ae03a84c, region_id=RegionOne) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:32:28.447901 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a551ba-da2b-4837-a2fa-1af048121f34 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:28.449278 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a551ba-da2b-4837-a2fa-1af048121f34 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:28.460502 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 508/1037] 10.222.0.89 () {56 vars in 1251 bytes} [Thu Jul 28 17:32:28 2022] GET /identity/v3/endpoints?service_id=bda3f4af2ff6410db475d094ae03a84c&interface=public®ion_id=RegionOne => generated 194 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:32:28.461870 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a551ba-da2b-4837-a2fa-1af048121f34 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:28.524287 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209b9d19-6125-4aa6-98a4-ce1f079d46d7 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:28.526268 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:28.564929 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe4d224-9021-493a-9547-587590502a93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:28.565582 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe4d224-9021-493a-9547-587590502a93 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:28.565862 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe4d224-9021-493a-9547-587590502a93 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:28.566134 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe4d224-9021-493a-9547-587590502a93 None admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:28.566914 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe4d224-9021-493a-9547-587590502a93 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:32:28.567217 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe4d224-9021-493a-9547-587590502a93 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:28.568635 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe4d224-9021-493a-9547-587590502a93 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:28.583346 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a551ba-da2b-4837-a2fa-1af048121f34 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:28.587371 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 530/1038] 10.222.0.89 () {56 vars in 1133 bytes} [Thu Jul 28 17:32:28 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:32:28.593139 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe4d224-9021-493a-9547-587590502a93 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:28.619873 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe45267-6962-424d-a83d-0f2d7f99bdc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:28.620495 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe45267-6962-424d-a83d-0f2d7f99bdc4 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:28.620772 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe45267-6962-424d-a83d-0f2d7f99bdc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:28.621314 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe45267-6962-424d-a83d-0f2d7f99bdc4 None admin] PATH_INFO: `/v3/services` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:28.621968 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe45267-6962-424d-a83d-0f2d7f99bdc4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:28.623871 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe45267-6962-424d-a83d-0f2d7f99bdc4 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:28.641188 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 509/1039] 10.222.0.89 () {56 vars in 1087 bytes} [Thu Jul 28 17:32:28 2022] GET /identity/v3/services => generated 1957 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:32:28.712000 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe4d224-9021-493a-9547-587590502a93 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:28.749466 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25d855a-1979-4f0b-857f-1c2a95cb0e6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:28.750110 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25d855a-1979-4f0b-857f-1c2a95cb0e6f None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:28.750390 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25d855a-1979-4f0b-857f-1c2a95cb0e6f None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:28.750662 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25d855a-1979-4f0b-857f-1c2a95cb0e6f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:28.782139 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:28.829226 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25d855a-1979-4f0b-857f-1c2a95cb0e6f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:28.830670 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25d855a-1979-4f0b-857f-1c2a95cb0e6f None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:28.847509 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6255e1cd-69ca-4b7b-8a55-782dfd895bab None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:28.847801 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6255e1cd-69ca-4b7b-8a55-782dfd895bab None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:28.848081 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6255e1cd-69ca-4b7b-8a55-782dfd895bab None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:28.849635 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 510/1040] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:32:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 17:32:28.854366 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd137cc-3626-4b38-9e40-c8e7ed0ae7a6 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:28.854649 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd137cc-3626-4b38-9e40-c8e7ed0ae7a6 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:28.854947 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd137cc-3626-4b38-9e40-c8e7ed0ae7a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:28.875616 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-acd137cc-3626-4b38-9e40-c8e7ed0ae7a6 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:32:28.877007 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-acd137cc-3626-4b38-9e40-c8e7ed0ae7a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:32:28.912937 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:28.944006 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 531/1041] 10.222.0.89 () {58 vars in 1282 bytes} [Thu Jul 28 17:32:28 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 28 17:32:28.946208 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 511/1042] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:32:28 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 17:32:28.949726 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25d855a-1979-4f0b-857f-1c2a95cb0e6f None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:28.971774 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd137cc-3626-4b38-9e40-c8e7ed0ae7a6 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:29.013349 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:29.059724 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25d855a-1979-4f0b-857f-1c2a95cb0e6f None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:29.082012 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd137cc-3626-4b38-9e40-c8e7ed0ae7a6 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:29.098592 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60413a2c-fe9c-48cb-a741-3d0e359aa7ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:29.099231 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60413a2c-fe9c-48cb-a741-3d0e359aa7ef None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:29.099510 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60413a2c-fe9c-48cb-a741-3d0e359aa7ef None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:29.099780 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60413a2c-fe9c-48cb-a741-3d0e359aa7ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:29.110784 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60413a2c-fe9c-48cb-a741-3d0e359aa7ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:29.112225 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60413a2c-fe9c-48cb-a741-3d0e359aa7ef None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:29.125386 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bdbf5d-46ca-422d-8123-9bc1923c54d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:29.126048 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bdbf5d-46ca-422d-8123-9bc1923c54d1 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:29.126330 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bdbf5d-46ca-422d-8123-9bc1923c54d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:29.126603 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bdbf5d-46ca-422d-8123-9bc1923c54d1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:29.128401 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-60413a2c-fe9c-48cb-a741-3d0e359aa7ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:32:29.128401 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:32:29.128401 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:32:29.128401 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:32:29.128401 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:32:29.128401 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:32:29.128401 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:32:29.128401 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:32:29.128401 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:32:29.128401 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:32:29.128401 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:32:29.128401 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:32:29.128401 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:32:29.128401 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:32:29.128401 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:32:29.129524 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:32:29.129524 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:32:29.129524 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:32:29.129524 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:32:29.129524 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:32:29.129524 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:32:29.129524 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:32:29.129524 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:32:29.129524 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( Jul 28 17:32:29.129524 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:32:29.129524 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:32:29.129524 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:32:29.129524 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:32:29.129524 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:32:29.129524 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:32:29.129524 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:32:29.129524 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:32:29.130429 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:32:29.130429 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:32:29.130429 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:32:29.130429 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:32:29.134338 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 532/1043] 10.222.0.89 () {56 vars in 1108 bytes} [Thu Jul 28 17:32:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:32:29.135482 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60413a2c-fe9c-48cb-a741-3d0e359aa7ef None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:29.137763 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bdbf5d-46ca-422d-8123-9bc1923c54d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:32:29.143001 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-41bdbf5d-46ca-422d-8123-9bc1923c54d1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:32:29.143001 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:32:29.143001 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:32:29.143001 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:32:29.143001 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:32:29.143001 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:32:29.143001 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:32:29.143001 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:32:29.143001 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:32:29.143001 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:32:29.143001 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:32:29.143001 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:32:29.143001 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:32:29.143001 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:32:29.143001 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:32:29.143001 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:32:29.143961 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:32:29.143961 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:32:29.143961 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:32:29.143961 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:32:29.143961 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:32:29.143961 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:32:29.143961 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:32:29.143961 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:32:29.143961 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:32:29.143961 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:32:29.143961 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:32:29.143961 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:32:29.143961 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:32:29.143961 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:32:29.143961 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:32:29.143961 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:32:29.146091 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:32:29.146091 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:32:29.146091 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:32:29.146091 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:32:29.146091 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:32:29.146091 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:32:29.146091 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:32:29.146091 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:32:29.146091 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:32:29.146091 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:32:29.146091 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:32:29.146091 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:32:29.146091 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 512/1044] 10.222.0.89 () {56 vars in 1096 bytes} [Thu Jul 28 17:32:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:32:29.150365 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bdbf5d-46ca-422d-8123-9bc1923c54d1 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:29.182211 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:29.251209 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60413a2c-fe9c-48cb-a741-3d0e359aa7ef None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:29.254397 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bdbf5d-46ca-422d-8123-9bc1923c54d1 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:29.266560 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:29.290406 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21333019-bd85-4623-a561-43281bc254b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:29.290983 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21333019-bd85-4623-a561-43281bc254b8 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:29.291258 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21333019-bd85-4623-a561-43281bc254b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:29.291527 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21333019-bd85-4623-a561-43281bc254b8 None admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:29.291753 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bfc7fd-7e85-4701-a08d-12d7757dfd01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:29.292209 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21333019-bd85-4623-a561-43281bc254b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:32:29.292370 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bfc7fd-7e85-4701-a08d-12d7757dfd01 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:29.292526 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21333019-bd85-4623-a561-43281bc254b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:29.292640 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bfc7fd-7e85-4701-a08d-12d7757dfd01 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:29.292894 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bfc7fd-7e85-4701-a08d-12d7757dfd01 None admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:29.293803 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bfc7fd-7e85-4701-a08d-12d7757dfd01 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:32:29.294093 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21333019-bd85-4623-a561-43281bc254b8 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:29.294210 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bfc7fd-7e85-4701-a08d-12d7757dfd01 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:29.296191 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bfc7fd-7e85-4701-a08d-12d7757dfd01 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:29.305568 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 513/1045] 10.222.0.89 () {56 vars in 1099 bytes} [Thu Jul 28 17:32:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:32:29.312897 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 533/1046] 10.222.0.89 () {56 vars in 1109 bytes} [Thu Jul 28 17:32:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:32:29.314030 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bfc7fd-7e85-4701-a08d-12d7757dfd01 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:29.314252 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21333019-bd85-4623-a561-43281bc254b8 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:29.414551 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:29.424375 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bfc7fd-7e85-4701-a08d-12d7757dfd01 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:29.428685 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21333019-bd85-4623-a561-43281bc254b8 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:29.462761 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00d8eda-7f7c-4f30-bf15-335df6390b80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:29.463502 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00d8eda-7f7c-4f30-bf15-335df6390b80 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:29.463694 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00d8eda-7f7c-4f30-bf15-335df6390b80 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:29.463970 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00d8eda-7f7c-4f30-bf15-335df6390b80 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:29.470964 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00d8eda-7f7c-4f30-bf15-335df6390b80 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:29.472410 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00d8eda-7f7c-4f30-bf15-335df6390b80 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:29.472879 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ba4cac-5696-45a6-9a70-3db18a1a145e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:29.473631 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ba4cac-5696-45a6-9a70-3db18a1a145e None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:29.473922 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ba4cac-5696-45a6-9a70-3db18a1a145e None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:29.474206 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ba4cac-5696-45a6-9a70-3db18a1a145e None admin] PATH_INFO: `/v3/users/1c079286ab384f4e86b21010b76f6fbf` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:29.484083 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-d00d8eda-7f7c-4f30-bf15-335df6390b80 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:32:29.484083 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:32:29.484083 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:32:29.484083 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:32:29.484083 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:32:29.484083 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:32:29.484083 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:32:29.484083 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:32:29.484083 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:32:29.484083 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:32:29.484083 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:32:29.484083 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:32:29.484083 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 17:32:29.484083 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 17:32:29.484083 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 17:32:29.485142 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 17:32:29.485142 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:32:29.485142 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:32:29.485142 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:32:29.485142 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:32:29.485142 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:32:29.485142 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:32:29.485142 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:32:29.485142 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:32:29.485142 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:32:29.485142 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:32:29.485142 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:32:29.485142 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:32:29.485142 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:32:29.485142 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:32:29.485142 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:32:29.485142 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:32:29.486086 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 17:32:29.486086 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 17:32:29.486086 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 17:32:29.486086 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 17:32:29.486086 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 17:32:29.486086 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 17:32:29.486086 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:32:29.486086 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:32:29.487304 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 514/1047] 10.222.0.89 () {56 vars in 1111 bytes} [Thu Jul 28 17:32:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:32:29.490323 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ba4cac-5696-45a6-9a70-3db18a1a145e None admin] RBAC: Authorizing `identity:get_user(user_id=1c079286ab384f4e86b21010b76f6fbf)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:29.491853 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ba4cac-5696-45a6-9a70-3db18a1a145e None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:29.492732 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00d8eda-7f7c-4f30-bf15-335df6390b80 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:29.498998 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 534/1048] 10.222.0.89 () {56 vars in 1177 bytes} [Thu Jul 28 17:32:29 2022] GET /identity/v3/users/1c079286ab384f4e86b21010b76f6fbf => generated 285 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:32:29.499998 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ba4cac-5696-45a6-9a70-3db18a1a145e None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:29.534554 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:29.606972 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00d8eda-7f7c-4f30-bf15-335df6390b80 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:29.607338 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ba4cac-5696-45a6-9a70-3db18a1a145e None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:29.644287 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdea647-e318-40f6-8836-a1a21a3cd5e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:29.644914 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdea647-e318-40f6-8836-a1a21a3cd5e3 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:29.645224 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdea647-e318-40f6-8836-a1a21a3cd5e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:29.645503 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdea647-e318-40f6-8836-a1a21a3cd5e3 None admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:29.646355 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdea647-e318-40f6-8836-a1a21a3cd5e3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:32:29.646770 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdea647-e318-40f6-8836-a1a21a3cd5e3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:29.647721 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8921f875-0d36-4268-b4da-52b561bcaa74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:29.648200 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdea647-e318-40f6-8836-a1a21a3cd5e3 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:29.648360 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8921f875-0d36-4268-b4da-52b561bcaa74 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:29.648618 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8921f875-0d36-4268-b4da-52b561bcaa74 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:29.648896 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8921f875-0d36-4268-b4da-52b561bcaa74 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:29.669526 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 515/1049] 10.222.0.89 () {56 vars in 1148 bytes} [Thu Jul 28 17:32:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:32:29.672839 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdea647-e318-40f6-8836-a1a21a3cd5e3 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:29.690835 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:29.725179 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8921f875-0d36-4268-b4da-52b561bcaa74 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:29.727502 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8921f875-0d36-4268-b4da-52b561bcaa74 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:29.768935 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:29.799460 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdea647-e318-40f6-8836-a1a21a3cd5e3 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:29.842340 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114353e5-bbb9-4b1d-bebb-ee5115885563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:29.843163 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114353e5-bbb9-4b1d-bebb-ee5115885563 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:29.843518 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114353e5-bbb9-4b1d-bebb-ee5115885563 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:29.843791 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114353e5-bbb9-4b1d-bebb-ee5115885563 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:29.844635 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114353e5-bbb9-4b1d-bebb-ee5115885563 None admin] RBAC: Adding query filter params (role.id=af4b58a9c88d4cb4861277524ab3e1ed, scope.project.id=04434e8535a64044996316f1d17dc0bc, user.id=d10b49b29df94404a71f43273839c252) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:32:29.844947 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114353e5-bbb9-4b1d-bebb-ee5115885563 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:29.846391 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114353e5-bbb9-4b1d-bebb-ee5115885563 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:29.852369 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 516/1050] 10.222.0.89 () {56 vars in 1374 bytes} [Thu Jul 28 17:32:29 2022] GET /identity/v3/role_assignments?user.id=d10b49b29df94404a71f43273839c252&scope.project.id=04434e8535a64044996316f1d17dc0bc&role.id=af4b58a9c88d4cb4861277524ab3e1ed => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:32:29.866811 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 535/1051] 10.222.0.89 () {58 vars in 1282 bytes} [Thu Jul 28 17:32:29 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 17:32:29.874256 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114353e5-bbb9-4b1d-bebb-ee5115885563 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:29.960163 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:29.987447 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114353e5-bbb9-4b1d-bebb-ee5115885563 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:30.025115 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d00c736-fd00-496e-822a-cc095bb4cb47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:30.025802 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d00c736-fd00-496e-822a-cc095bb4cb47 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:30.026074 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d00c736-fd00-496e-822a-cc095bb4cb47 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:30.026347 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d00c736-fd00-496e-822a-cc095bb4cb47 None admin] PATH_INFO: `/v3/projects/9face3e074924844aff9ccb35b7ca307` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:30.027507 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d00c736-fd00-496e-822a-cc095bb4cb47 None admin] RBAC: Authorizing `identity:get_project(project_id=9face3e074924844aff9ccb35b7ca307)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:30.028936 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d00c736-fd00-496e-822a-cc095bb4cb47 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:30.031908 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 517/1052] 10.222.0.89 () {56 vars in 1186 bytes} [Thu Jul 28 17:32:29 2022] GET /identity/v3/projects/9face3e074924844aff9ccb35b7ca307 => generated 311 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:32:30.038125 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8921f875-0d36-4268-b4da-52b561bcaa74 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:30.142153 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8921f875-0d36-4268-b4da-52b561bcaa74 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:30.171285 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:30.178188 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5adab52-a5e2-4628-859d-d6e236544b4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:30.178825 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5adab52-a5e2-4628-859d-d6e236544b4d None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:30.179108 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5adab52-a5e2-4628-859d-d6e236544b4d None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:30.179382 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5adab52-a5e2-4628-859d-d6e236544b4d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:30.180163 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5adab52-a5e2-4628-859d-d6e236544b4d None admin] RBAC: Adding query filter params (role.id=73e457231a824ea98c28d22610be640c, scope.project.id=9face3e074924844aff9ccb35b7ca307, user.id=1c079286ab384f4e86b21010b76f6fbf) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:32:30.180474 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5adab52-a5e2-4628-859d-d6e236544b4d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:30.181928 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5adab52-a5e2-4628-859d-d6e236544b4d None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:30.193195 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 536/1053] 10.222.0.89 () {56 vars in 1374 bytes} [Thu Jul 28 17:32:30 2022] GET /identity/v3/role_assignments?user.id=1c079286ab384f4e86b21010b76f6fbf&scope.project.id=9face3e074924844aff9ccb35b7ca307&role.id=73e457231a824ea98c28d22610be640c => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:32:30.704023 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf86dde-c536-49b8-9146-f24f7af627ae None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:30.704563 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf86dde-c536-49b8-9146-f24f7af627ae None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:30.704563 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf86dde-c536-49b8-9146-f24f7af627ae None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:30.705982 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 518/1054] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:32:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:32:30.711130 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4169fb6-ec67-4c78-942f-c6e5183af9a2 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:30.711548 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4169fb6-ec67-4c78-942f-c6e5183af9a2 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:30.711689 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4169fb6-ec67-4c78-942f-c6e5183af9a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:30.730442 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-b4169fb6-ec67-4c78-942f-c6e5183af9a2 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:32:30.731645 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-b4169fb6-ec67-4c78-942f-c6e5183af9a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:32:30.801398 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 537/1055] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:32:30 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:32:30.828403 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf86dde-c536-49b8-9146-f24f7af627ae None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:30.932990 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf86dde-c536-49b8-9146-f24f7af627ae None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:30.967497 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9f3aa2-53af-4f76-8bec-bac1be1c149a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:30.968053 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9f3aa2-53af-4f76-8bec-bac1be1c149a None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:30.968333 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9f3aa2-53af-4f76-8bec-bac1be1c149a None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:30.968604 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9f3aa2-53af-4f76-8bec-bac1be1c149a None admin] PATH_INFO: `/v3/services/image` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:30.969283 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9f3aa2-53af-4f76-8bec-bac1be1c149a None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:30.970692 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9f3aa2-53af-4f76-8bec-bac1be1c149a None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:30.980635 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-4b9f3aa2-53af-4f76-8bec-bac1be1c149a None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 17:32:30.980635 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:32:30.980635 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 17:32:30.980635 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 17:32:30.980635 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 17:32:30.980635 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 17:32:30.980635 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 17:32:30.980635 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:32:30.980635 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 17:32:30.980635 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:32:30.980635 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 17:32:30.980635 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:32:30.980635 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:32:30.980635 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:32:30.980635 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:32:30.980635 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:32:30.980635 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:32:30.982003 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:32:30.982003 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:32:30.982003 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:32:30.982003 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:32:30.982003 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:32:30.982003 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:32:30.982003 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 17:32:30.982003 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 17:32:30.982003 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 17:32:30.982003 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 17:32:30.982003 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:32:30.982003 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:32:30.982003 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:32:30.982003 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:32:30.982003 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:32:30.982003 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:32:30.983123 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:32:30.983123 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:32:30.983123 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:32:30.983123 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:32:30.983123 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:32:30.983123 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:32:30.983123 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:32:30.983123 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:32:30.983123 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:32:30.983123 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:32:30.983123 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 17:32:30.983123 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:32:30.983123 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 17:32:30.983123 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:32:30.983123 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 519/1056] 10.222.0.89 () {56 vars in 1105 bytes} [Thu Jul 28 17:32:30 2022] GET /identity/v3/services/image => generated 86 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:32:30.988272 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4169fb6-ec67-4c78-942f-c6e5183af9a2 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:31.095575 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4169fb6-ec67-4c78-942f-c6e5183af9a2 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:31.118755 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:31.131973 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e50ecb9-f9e6-4970-bd1a-86f6e064aeaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:31.132593 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e50ecb9-f9e6-4970-bd1a-86f6e064aeaa None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:31.132875 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e50ecb9-f9e6-4970-bd1a-86f6e064aeaa None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:31.133150 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e50ecb9-f9e6-4970-bd1a-86f6e064aeaa None admin] PATH_INFO: `/v3/services` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:31.133863 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e50ecb9-f9e6-4970-bd1a-86f6e064aeaa None admin] RBAC: Adding query filter params (name=image) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:32:31.134171 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e50ecb9-f9e6-4970-bd1a-86f6e064aeaa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:31.135535 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e50ecb9-f9e6-4970-bd1a-86f6e064aeaa None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:31.141196 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 538/1057] 10.222.0.89 () {56 vars in 1108 bytes} [Thu Jul 28 17:32:30 2022] GET /identity/v3/services?name=image => generated 122 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:32:31.147109 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9f3aa2-53af-4f76-8bec-bac1be1c149a None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:31.158398 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:31.251124 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9f3aa2-53af-4f76-8bec-bac1be1c149a None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:31.286418 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef1a617-3177-4faa-8970-ceb282bcce46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:31.287059 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef1a617-3177-4faa-8970-ceb282bcce46 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:31.287342 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef1a617-3177-4faa-8970-ceb282bcce46 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:31.287619 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef1a617-3177-4faa-8970-ceb282bcce46 None admin] PATH_INFO: `/v3/services` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:31.288321 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef1a617-3177-4faa-8970-ceb282bcce46 None admin] RBAC: Adding query filter params (type=image) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:32:31.288684 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef1a617-3177-4faa-8970-ceb282bcce46 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:31.290098 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef1a617-3177-4faa-8970-ceb282bcce46 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:31.301377 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 520/1058] 10.222.0.89 () {56 vars in 1108 bytes} [Thu Jul 28 17:32:31 2022] GET /identity/v3/services?type=image => generated 350 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:32:31.303062 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e50ecb9-f9e6-4970-bd1a-86f6e064aeaa None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:31.408304 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e50ecb9-f9e6-4970-bd1a-86f6e064aeaa None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:31.443280 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a0b3ea-bfcc-4088-9843-a77c417b9d38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:31.443893 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a0b3ea-bfcc-4088-9843-a77c417b9d38 None admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:31.444171 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a0b3ea-bfcc-4088-9843-a77c417b9d38 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:31.444442 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a0b3ea-bfcc-4088-9843-a77c417b9d38 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:31.445232 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a0b3ea-bfcc-4088-9843-a77c417b9d38 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:31.446538 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a0b3ea-bfcc-4088-9843-a77c417b9d38 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:31.469025 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 539/1059] 10.222.0.89 () {60 vars in 1144 bytes} [Thu Jul 28 17:32:31 2022] POST /identity/v3/endpoints => generated 327 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:32:31.911686 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6d583c-7203-488f-950c-6395291a8858 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:31.912198 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6d583c-7203-488f-950c-6395291a8858 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:31.912277 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6d583c-7203-488f-950c-6395291a8858 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:31.913746 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 521/1060] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:32:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:32:31.918612 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd4a836-f0bf-4623-b301-2f485096a8f4 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:31.918992 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd4a836-f0bf-4623-b301-2f485096a8f4 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:31.919207 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd4a836-f0bf-4623-b301-2f485096a8f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:31.938898 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-7dd4a836-f0bf-4623-b301-2f485096a8f4 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:32:31.940091 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-7dd4a836-f0bf-4623-b301-2f485096a8f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:32:31.983536 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:32.020722 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 540/1061] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:32:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:32:32.043149 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6d583c-7203-488f-950c-6395291a8858 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:32.116332 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:32.150051 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6d583c-7203-488f-950c-6395291a8858 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:32.186145 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c573a66a-e6a9-4254-ab1f-a581a302e644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:32.187241 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c573a66a-e6a9-4254-ab1f-a581a302e644 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:32.187515 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c573a66a-e6a9-4254-ab1f-a581a302e644 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:32.187780 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c573a66a-e6a9-4254-ab1f-a581a302e644 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:32.198348 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c573a66a-e6a9-4254-ab1f-a581a302e644 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:32:32.200516 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febeb65b-7374-47ee-a733-2ab114dc14ad None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:32.200810 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febeb65b-7374-47ee-a733-2ab114dc14ad None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:32.201137 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febeb65b-7374-47ee-a733-2ab114dc14ad None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:32.202863 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 541/1062] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:32:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:32:32.203925 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-c573a66a-e6a9-4254-ab1f-a581a302e644 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 17:32:32.203925 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:32:32.203925 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:32:32.203925 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:32:32.203925 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:32:32.203925 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:32:32.203925 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:32:32.203925 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:32:32.203925 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:32:32.203925 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:32:32.203925 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:32:32.203925 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:32:32.203925 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:32:32.203925 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:32:32.203925 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:32:32.203925 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:32:32.204853 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:32:32.204853 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:32:32.204853 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:32:32.204853 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:32:32.204853 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:32:32.204853 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:32:32.204853 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:32:32.204853 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:32:32.204853 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:32:32.204853 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:32:32.204853 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:32:32.204853 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:32:32.204853 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:32:32.204853 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:32:32.204853 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:32:32.204853 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:32:32.206070 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:32:32.206070 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:32:32.206070 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:32:32.206070 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:32:32.206070 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:32:32.206070 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:32:32.206070 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:32:32.206070 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:32:32.206070 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:32:32.206070 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:32:32.206070 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 17:32:32.206070 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:32:32.207812 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcef74bb-43f5-4ba3-b755-9bf2f54e8e59 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:32.208100 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcef74bb-43f5-4ba3-b755-9bf2f54e8e59 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:32.208379 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcef74bb-43f5-4ba3-b755-9bf2f54e8e59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:32.209452 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 522/1063] 10.222.0.89 () {56 vars in 1120 bytes} [Thu Jul 28 17:32:32 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:32:32.210745 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c573a66a-e6a9-4254-ab1f-a581a302e644 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:32.227661 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-fcef74bb-43f5-4ba3-b755-9bf2f54e8e59 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:32:32.228881 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-fcef74bb-43f5-4ba3-b755-9bf2f54e8e59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:32:32.302453 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 542/1064] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:32:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 17:32:32.324717 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c573a66a-e6a9-4254-ab1f-a581a302e644 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:32.326820 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcef74bb-43f5-4ba3-b755-9bf2f54e8e59 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:32.364324 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:32.370794 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec060a93-89a3-46c6-b72d-166b16465d78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:32.371968 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec060a93-89a3-46c6-b72d-166b16465d78 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:32.372571 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec060a93-89a3-46c6-b72d-166b16465d78 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:32.372963 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec060a93-89a3-46c6-b72d-166b16465d78 None admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:32.373726 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec060a93-89a3-46c6-b72d-166b16465d78 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:32:32.374036 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec060a93-89a3-46c6-b72d-166b16465d78 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:32.375414 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec060a93-89a3-46c6-b72d-166b16465d78 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:32.384170 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 523/1065] 10.222.0.89 () {56 vars in 1115 bytes} [Thu Jul 28 17:32:32 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:32:32.390818 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec060a93-89a3-46c6-b72d-166b16465d78 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:32.441989 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:32.445582 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcef74bb-43f5-4ba3-b755-9bf2f54e8e59 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:32.483779 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052c6bab-fdc1-459c-9a58-568da9ff0e82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:32.484536 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052c6bab-fdc1-459c-9a58-568da9ff0e82 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:32.484815 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052c6bab-fdc1-459c-9a58-568da9ff0e82 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:32.485086 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052c6bab-fdc1-459c-9a58-568da9ff0e82 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:32.496456 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052c6bab-fdc1-459c-9a58-568da9ff0e82 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:32:32.502376 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-052c6bab-fdc1-459c-9a58-568da9ff0e82 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:32:32.502376 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:32:32.502376 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:32:32.502376 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:32:32.502376 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:32:32.502376 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:32:32.502376 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:32:32.502376 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:32:32.502376 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:32:32.502376 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:32:32.502376 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:32:32.502376 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:32:32.502376 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:32:32.502376 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:32:32.502376 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:32:32.502376 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:32:32.503482 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:32:32.503482 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:32:32.503482 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:32:32.503482 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:32:32.503482 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:32:32.503482 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:32:32.503482 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:32:32.503482 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:32:32.503482 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:32:32.503482 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:32:32.503482 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:32:32.503482 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( Jul 28 17:32:32.503482 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:32:32.503482 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:32:32.503482 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:32:32.503482 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:32:32.504839 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:32:32.504839 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:32:32.504839 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:32:32.504839 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:32:32.504839 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:32:32.504839 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:32:32.504839 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:32:32.504839 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:32:32.504839 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:32:32.504839 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:32:32.504839 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:32:32.504839 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:32:32.504839 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 543/1066] 10.222.0.89 () {56 vars in 1096 bytes} [Thu Jul 28 17:32:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:32:32.509513 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052c6bab-fdc1-459c-9a58-568da9ff0e82 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:32.514731 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec060a93-89a3-46c6-b72d-166b16465d78 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:32.549318 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4b15fa-8fed-498e-96f7-b90324f0931a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:32.549970 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4b15fa-8fed-498e-96f7-b90324f0931a None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:32.550254 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4b15fa-8fed-498e-96f7-b90324f0931a None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:32.550528 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4b15fa-8fed-498e-96f7-b90324f0931a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:32.562085 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4b15fa-8fed-498e-96f7-b90324f0931a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:32.563962 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4b15fa-8fed-498e-96f7-b90324f0931a None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:32.578841 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-dc4b15fa-8fed-498e-96f7-b90324f0931a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:32:32.578841 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:32:32.578841 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:32:32.578841 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:32:32.578841 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:32:32.578841 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:32:32.578841 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:32:32.578841 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:32:32.578841 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:32:32.578841 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:32:32.578841 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:32:32.578841 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:32:32.578841 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:32:32.578841 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:32:32.578841 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:32:32.579796 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:32:32.579796 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:32:32.579796 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:32:32.579796 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:32:32.579796 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:32:32.579796 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:32:32.579796 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:32:32.579796 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:32:32.579796 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:32:32.579796 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:32:32.579796 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:32:32.579796 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:32:32.579796 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:32:32.579796 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:32:32.579796 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:32:32.579796 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:32:32.579796 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:32:32.581217 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:32:32.581217 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:32:32.581217 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:32:32.581217 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:32:32.581217 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 524/1067] 10.222.0.89 () {56 vars in 1108 bytes} [Thu Jul 28 17:32:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:32:32.585582 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4b15fa-8fed-498e-96f7-b90324f0931a None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:32.627211 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052c6bab-fdc1-459c-9a58-568da9ff0e82 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:32.646120 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:32.664590 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1a2127-3fdb-44de-9c3c-c6b8c40a030a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:32.665234 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1a2127-3fdb-44de-9c3c-c6b8c40a030a None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:32.665521 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1a2127-3fdb-44de-9c3c-c6b8c40a030a None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:32.665797 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1a2127-3fdb-44de-9c3c-c6b8c40a030a None admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:32.666565 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1a2127-3fdb-44de-9c3c-c6b8c40a030a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:32:32.666869 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1a2127-3fdb-44de-9c3c-c6b8c40a030a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:32.668241 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1a2127-3fdb-44de-9c3c-c6b8c40a030a None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:32.677002 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 544/1068] 10.222.0.89 () {56 vars in 1099 bytes} [Thu Jul 28 17:32:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:32:32.683108 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1a2127-3fdb-44de-9c3c-c6b8c40a030a None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:32.707150 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4b15fa-8fed-498e-96f7-b90324f0931a None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:32.714569 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:32.744915 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001e7b6d-732f-4818-83c3-09ceeacb6736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:32.745570 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001e7b6d-732f-4818-83c3-09ceeacb6736 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:32.745853 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001e7b6d-732f-4818-83c3-09ceeacb6736 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:32.746128 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001e7b6d-732f-4818-83c3-09ceeacb6736 None admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:32.747081 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001e7b6d-732f-4818-83c3-09ceeacb6736 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:32:32.747683 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001e7b6d-732f-4818-83c3-09ceeacb6736 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:32.749379 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001e7b6d-732f-4818-83c3-09ceeacb6736 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:32.761942 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 525/1069] 10.222.0.89 () {56 vars in 1109 bytes} [Thu Jul 28 17:32:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:32:32.767771 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001e7b6d-732f-4818-83c3-09ceeacb6736 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:32.806422 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1a2127-3fdb-44de-9c3c-c6b8c40a030a None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:32.851028 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfa1575-54bd-415c-89e5-a82b9e159e79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:32.851659 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfa1575-54bd-415c-89e5-a82b9e159e79 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:32.851940 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfa1575-54bd-415c-89e5-a82b9e159e79 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:32.852214 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfa1575-54bd-415c-89e5-a82b9e159e79 None admin] PATH_INFO: `/v3/users/1c079286ab384f4e86b21010b76f6fbf` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:32.853457 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfa1575-54bd-415c-89e5-a82b9e159e79 None admin] RBAC: Authorizing `identity:get_user(user_id=1c079286ab384f4e86b21010b76f6fbf)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:32.854881 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfa1575-54bd-415c-89e5-a82b9e159e79 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:32.856643 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 545/1070] 10.222.0.89 () {56 vars in 1177 bytes} [Thu Jul 28 17:32:32 2022] GET /identity/v3/users/1c079286ab384f4e86b21010b76f6fbf => generated 285 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:32:32.862255 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfa1575-54bd-415c-89e5-a82b9e159e79 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:32.881672 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001e7b6d-732f-4818-83c3-09ceeacb6736 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:32.889661 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:32.918958 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835478a7-5f27-4cbd-b797-98715b43c468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:32.919612 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835478a7-5f27-4cbd-b797-98715b43c468 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:32.919905 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835478a7-5f27-4cbd-b797-98715b43c468 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:32.920179 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835478a7-5f27-4cbd-b797-98715b43c468 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:32.930011 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835478a7-5f27-4cbd-b797-98715b43c468 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:32.932176 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835478a7-5f27-4cbd-b797-98715b43c468 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:32.947193 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-835478a7-5f27-4cbd-b797-98715b43c468 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 17:32:32.947193 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:32:32.947193 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:32:32.947193 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:32:32.947193 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:32:32.947193 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:32:32.947193 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:32:32.947193 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:32:32.947193 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:32:32.947193 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:32:32.947193 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:32:32.947193 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:32:32.947193 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 17:32:32.947193 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 17:32:32.947193 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 17:32:32.947193 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 17:32:32.948244 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:32:32.948244 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:32:32.948244 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 17:32:32.948244 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:32:32.948244 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 17:32:32.948244 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:32:32.948244 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:32:32.948244 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:32:32.948244 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:32:32.948244 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:32:32.948244 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:32:32.948244 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:32:32.948244 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:32:32.948244 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:32:32.948244 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:32:32.948244 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:32:32.948244 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:32:32.948244 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:32:32.949402 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:32:32.949402 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:32:32.949402 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 17:32:32.949402 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 17:32:32.949402 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 17:32:32.949402 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 17:32:32.949402 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 17:32:32.949402 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 17:32:32.949402 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 17:32:32.949402 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:32:32.956647 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 526/1071] 10.222.0.89 () {56 vars in 1093 bytes} [Thu Jul 28 17:32:32 2022] GET /identity/v3/users/nova => generated 82 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:32:32.957719 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835478a7-5f27-4cbd-b797-98715b43c468 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:32.962838 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:32.975337 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfa1575-54bd-415c-89e5-a82b9e159e79 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:33.013761 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659a8b2d-d34f-401d-b28b-038eadbba227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:33.014563 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659a8b2d-d34f-401d-b28b-038eadbba227 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:33.014851 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659a8b2d-d34f-401d-b28b-038eadbba227 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:33.015154 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659a8b2d-d34f-401d-b28b-038eadbba227 None admin] PATH_INFO: `/v3/projects/9face3e074924844aff9ccb35b7ca307` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:33.016457 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659a8b2d-d34f-401d-b28b-038eadbba227 None admin] RBAC: Authorizing `identity:get_project(project_id=9face3e074924844aff9ccb35b7ca307)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:33.018047 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659a8b2d-d34f-401d-b28b-038eadbba227 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:33.024597 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 546/1072] 10.222.0.89 () {56 vars in 1186 bytes} [Thu Jul 28 17:32:32 2022] GET /identity/v3/projects/9face3e074924844aff9ccb35b7ca307 => generated 311 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:32:33.026007 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659a8b2d-d34f-401d-b28b-038eadbba227 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:33.068378 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835478a7-5f27-4cbd-b797-98715b43c468 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:33.108546 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7728cc8d-0d95-4cc3-a9a8-324800827b09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:33.109519 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7728cc8d-0d95-4cc3-a9a8-324800827b09 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:33.109876 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7728cc8d-0d95-4cc3-a9a8-324800827b09 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:33.110150 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7728cc8d-0d95-4cc3-a9a8-324800827b09 None admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:33.111022 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7728cc8d-0d95-4cc3-a9a8-324800827b09 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:32:33.111333 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7728cc8d-0d95-4cc3-a9a8-324800827b09 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:33.112762 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7728cc8d-0d95-4cc3-a9a8-324800827b09 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:33.135737 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 527/1073] 10.222.0.89 () {56 vars in 1133 bytes} [Thu Jul 28 17:32:32 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:32:33.136694 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7728cc8d-0d95-4cc3-a9a8-324800827b09 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:33.147244 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659a8b2d-d34f-401d-b28b-038eadbba227 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:33.169849 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:33.195280 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dfb82b-e368-45c1-bb31-7628cd00e3df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:33.195954 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dfb82b-e368-45c1-bb31-7628cd00e3df None admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:33.196238 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dfb82b-e368-45c1-bb31-7628cd00e3df None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:33.196518 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dfb82b-e368-45c1-bb31-7628cd00e3df None admin] PATH_INFO: `/v3/projects/9face3e074924844aff9ccb35b7ca307/users/1c079286ab384f4e86b21010b76f6fbf/roles/73e457231a824ea98c28d22610be640c` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:33.198547 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dfb82b-e368-45c1-bb31-7628cd00e3df None admin] RBAC: Authorizing `identity:create_grant(project_id=9face3e074924844aff9ccb35b7ca307, user_id=1c079286ab384f4e86b21010b76f6fbf, role_id=73e457231a824ea98c28d22610be640c)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:33.199992 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dfb82b-e368-45c1-bb31-7628cd00e3df None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:33.215651 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 547/1074] 10.222.0.89 () {58 vars in 1439 bytes} [Thu Jul 28 17:32:33 2022] PUT /identity/v3/projects/9face3e074924844aff9ccb35b7ca307/users/1c079286ab384f4e86b21010b76f6fbf/roles/73e457231a824ea98c28d22610be640c => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 17:32:33.267832 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7728cc8d-0d95-4cc3-a9a8-324800827b09 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:33.306751 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3687a0-234b-4a04-b0ec-97b2854e33b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:33.307424 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3687a0-234b-4a04-b0ec-97b2854e33b1 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:33.307703 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3687a0-234b-4a04-b0ec-97b2854e33b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:33.307975 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3687a0-234b-4a04-b0ec-97b2854e33b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:33.320201 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3687a0-234b-4a04-b0ec-97b2854e33b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:33.321777 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3687a0-234b-4a04-b0ec-97b2854e33b1 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:33.338897 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-4c3687a0-234b-4a04-b0ec-97b2854e33b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:32:33.338897 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:32:33.338897 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:32:33.338897 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:32:33.338897 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:32:33.338897 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:32:33.338897 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:32:33.338897 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:32:33.338897 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:32:33.338897 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:32:33.338897 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:32:33.338897 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:32:33.338897 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:32:33.338897 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:32:33.338897 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:32:33.339889 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:32:33.339889 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:32:33.339889 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:32:33.339889 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:32:33.339889 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:32:33.339889 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:32:33.339889 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:32:33.339889 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:32:33.339889 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:32:33.339889 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:32:33.339889 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:32:33.339889 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:32:33.339889 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:32:33.339889 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:32:33.339889 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:32:33.339889 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:32:33.339889 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:32:33.341496 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:32:33.341496 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:32:33.341496 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:32:33.341496 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:32:33.341496 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 528/1075] 10.222.0.89 () {56 vars in 1108 bytes} [Thu Jul 28 17:32:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:32:33.346639 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dfb82b-e368-45c1-bb31-7628cd00e3df None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:33.368132 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:33.457442 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dfb82b-e368-45c1-bb31-7628cd00e3df None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:33.491368 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0dc8c4-8dd8-4dbf-ac2c-b89a8321a820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:33.491979 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0dc8c4-8dd8-4dbf-ac2c-b89a8321a820 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:33.492262 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0dc8c4-8dd8-4dbf-ac2c-b89a8321a820 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:33.492543 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0dc8c4-8dd8-4dbf-ac2c-b89a8321a820 None admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:33.493267 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0dc8c4-8dd8-4dbf-ac2c-b89a8321a820 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:32:33.493576 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0dc8c4-8dd8-4dbf-ac2c-b89a8321a820 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:33.494946 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0dc8c4-8dd8-4dbf-ac2c-b89a8321a820 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:33.508117 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 548/1076] 10.222.0.89 () {56 vars in 1109 bytes} [Thu Jul 28 17:32:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:32:33.514525 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3687a0-234b-4a04-b0ec-97b2854e33b1 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:33.566516 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80696720-6afc-4efd-99d3-adc79ec0de56 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:33.566842 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80696720-6afc-4efd-99d3-adc79ec0de56 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:33.567070 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80696720-6afc-4efd-99d3-adc79ec0de56 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:33.568526 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 549/1077] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:32:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 17:32:33.573189 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f917ff6a-1b40-4104-adff-0f4787b9927a None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:33.573530 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f917ff6a-1b40-4104-adff-0f4787b9927a None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:33.573726 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f917ff6a-1b40-4104-adff-0f4787b9927a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:33.593924 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:33.601016 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-f917ff6a-1b40-4104-adff-0f4787b9927a None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:32:33.602304 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-f917ff6a-1b40-4104-adff-0f4787b9927a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:32:33.623673 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3687a0-234b-4a04-b0ec-97b2854e33b1 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:33.660497 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84476e7-007e-473f-b664-53bf26c7e631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:33.661148 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84476e7-007e-473f-b664-53bf26c7e631 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:33.661457 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84476e7-007e-473f-b664-53bf26c7e631 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:33.661731 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84476e7-007e-473f-b664-53bf26c7e631 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:33.668561 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84476e7-007e-473f-b664-53bf26c7e631 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:33.670714 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84476e7-007e-473f-b664-53bf26c7e631 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:33.678652 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 550/1078] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:32:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 17:32:33.682772 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-c84476e7-007e-473f-b664-53bf26c7e631 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:32:33.682772 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:32:33.682772 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:32:33.682772 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:32:33.682772 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:32:33.682772 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:32:33.682772 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:32:33.682772 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:32:33.682772 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:32:33.682772 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:32:33.682772 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:32:33.682772 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:32:33.682772 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 17:32:33.682772 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 17:32:33.682772 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 17:32:33.683667 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 17:32:33.683667 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:32:33.683667 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:32:33.683667 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:32:33.683667 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:32:33.683667 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:32:33.683667 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:32:33.683667 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:32:33.683667 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:32:33.683667 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:32:33.683667 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:32:33.683667 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:32:33.683667 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:32:33.683667 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:32:33.683667 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:32:33.683667 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:32:33.683667 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:32:33.684572 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 17:32:33.684572 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 17:32:33.684572 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 17:32:33.684572 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 17:32:33.684572 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 17:32:33.684572 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 17:32:33.684572 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:32:33.684572 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:32:33.689494 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 529/1079] 10.222.0.89 () {56 vars in 1111 bytes} [Thu Jul 28 17:32:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:32:33.690802 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f917ff6a-1b40-4104-adff-0f4787b9927a None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:33.701836 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84476e7-007e-473f-b664-53bf26c7e631 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:33.702258 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:33.803664 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f917ff6a-1b40-4104-adff-0f4787b9927a None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:33.816456 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84476e7-007e-473f-b664-53bf26c7e631 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:33.845860 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cc9a5a-a461-4894-ad76-6ce424ffe7c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:33.846489 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cc9a5a-a461-4894-ad76-6ce424ffe7c8 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:33.847138 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cc9a5a-a461-4894-ad76-6ce424ffe7c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:33.847138 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cc9a5a-a461-4894-ad76-6ce424ffe7c8 None admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:33.847867 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cc9a5a-a461-4894-ad76-6ce424ffe7c8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:32:33.848085 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cc9a5a-a461-4894-ad76-6ce424ffe7c8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:33.849545 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cc9a5a-a461-4894-ad76-6ce424ffe7c8 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:33.859142 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30430e44-5187-4ddc-92aa-b64b06dfb324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:33.859458 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30430e44-5187-4ddc-92aa-b64b06dfb324 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:33.859736 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30430e44-5187-4ddc-92aa-b64b06dfb324 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:33.860005 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30430e44-5187-4ddc-92aa-b64b06dfb324 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:33.867453 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 551/1080] 10.222.0.89 () {56 vars in 1148 bytes} [Thu Jul 28 17:32:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:32:33.868339 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cc9a5a-a461-4894-ad76-6ce424ffe7c8 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:33.869744 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:33.940186 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30430e44-5187-4ddc-92aa-b64b06dfb324 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:33.940846 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:33.942588 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30430e44-5187-4ddc-92aa-b64b06dfb324 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:33.976585 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cc9a5a-a461-4894-ad76-6ce424ffe7c8 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:34.014537 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612987ba-aac5-437a-ad35-6eea046c10eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:34.015300 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612987ba-aac5-437a-ad35-6eea046c10eb None admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:34.015675 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612987ba-aac5-437a-ad35-6eea046c10eb None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:34.015960 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612987ba-aac5-437a-ad35-6eea046c10eb None admin] PATH_INFO: `/v3/projects/04434e8535a64044996316f1d17dc0bc/users/d10b49b29df94404a71f43273839c252/roles/af4b58a9c88d4cb4861277524ab3e1ed` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:34.018263 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612987ba-aac5-437a-ad35-6eea046c10eb None admin] RBAC: Authorizing `identity:create_grant(project_id=04434e8535a64044996316f1d17dc0bc, user_id=d10b49b29df94404a71f43273839c252, role_id=af4b58a9c88d4cb4861277524ab3e1ed)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:34.019708 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612987ba-aac5-437a-ad35-6eea046c10eb None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:34.034848 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 552/1081] 10.222.0.89 () {58 vars in 1439 bytes} [Thu Jul 28 17:32:33 2022] PUT /identity/v3/projects/04434e8535a64044996316f1d17dc0bc/users/d10b49b29df94404a71f43273839c252/roles/af4b58a9c88d4cb4861277524ab3e1ed => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 17:32:34.055334 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 530/1082] 10.222.0.89 () {58 vars in 1282 bytes} [Thu Jul 28 17:32:33 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 28 17:32:34.057708 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612987ba-aac5-437a-ad35-6eea046c10eb None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:34.178181 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612987ba-aac5-437a-ad35-6eea046c10eb None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:34.208495 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:34.220216 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d244392b-6338-45d1-9aaf-7145b62e47b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:34.220869 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d244392b-6338-45d1-9aaf-7145b62e47b0 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:34.221152 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d244392b-6338-45d1-9aaf-7145b62e47b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:34.221453 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d244392b-6338-45d1-9aaf-7145b62e47b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:34.233277 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d244392b-6338-45d1-9aaf-7145b62e47b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:34.234726 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d244392b-6338-45d1-9aaf-7145b62e47b0 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:34.249664 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-d244392b-6338-45d1-9aaf-7145b62e47b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:32:34.249664 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:32:34.249664 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:32:34.249664 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:32:34.249664 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:32:34.249664 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:32:34.249664 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:32:34.249664 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:32:34.249664 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:32:34.249664 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:32:34.249664 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:32:34.249664 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:32:34.249664 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:32:34.249664 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:32:34.249664 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:32:34.250579 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:32:34.250579 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:32:34.250579 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:32:34.250579 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:32:34.250579 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:32:34.250579 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:32:34.250579 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:32:34.250579 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:32:34.250579 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( Jul 28 17:32:34.250579 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:32:34.250579 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:32:34.250579 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:32:34.250579 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:32:34.250579 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:32:34.250579 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:32:34.250579 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:32:34.250579 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:32:34.255542 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:32:34.255542 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:32:34.255542 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:32:34.255542 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:32:34.255542 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 553/1083] 10.222.0.89 () {56 vars in 1108 bytes} [Thu Jul 28 17:32:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:32:34.257401 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30430e44-5187-4ddc-92aa-b64b06dfb324 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:34.292821 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:34.360479 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30430e44-5187-4ddc-92aa-b64b06dfb324 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:34.397332 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3b7a43-4536-4786-a43a-34de8d3ad130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:34.397949 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3b7a43-4536-4786-a43a-34de8d3ad130 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:34.398226 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3b7a43-4536-4786-a43a-34de8d3ad130 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:34.398495 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3b7a43-4536-4786-a43a-34de8d3ad130 None admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:34.399190 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3b7a43-4536-4786-a43a-34de8d3ad130 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:32:34.399492 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3b7a43-4536-4786-a43a-34de8d3ad130 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:34.400850 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3b7a43-4536-4786-a43a-34de8d3ad130 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:34.421199 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 531/1084] 10.222.0.89 () {56 vars in 1109 bytes} [Thu Jul 28 17:32:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:32:35.269647 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87daea4-dd34-41e2-93ee-49e8b2d71e4f None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:35.270183 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87daea4-dd34-41e2-93ee-49e8b2d71e4f None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:35.270278 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87daea4-dd34-41e2-93ee-49e8b2d71e4f None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:35.271545 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 554/1085] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:32:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:32:35.276679 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbe18f9-1918-4762-b0d3-1e611263d318 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:35.277101 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbe18f9-1918-4762-b0d3-1e611263d318 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:35.277286 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbe18f9-1918-4762-b0d3-1e611263d318 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:35.296011 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-4bbe18f9-1918-4762-b0d3-1e611263d318 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:32:35.297234 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-4bbe18f9-1918-4762-b0d3-1e611263d318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:32:35.363070 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 532/1086] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:32:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 17:32:35.386663 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87daea4-dd34-41e2-93ee-49e8b2d71e4f None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:35.488202 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87daea4-dd34-41e2-93ee-49e8b2d71e4f None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:35.523237 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9f135b-1aa2-4bd4-a3f5-ba4cc5e1de8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:35.523792 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9f135b-1aa2-4bd4-a3f5-ba4cc5e1de8f None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:35.524073 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9f135b-1aa2-4bd4-a3f5-ba4cc5e1de8f None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:35.524345 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9f135b-1aa2-4bd4-a3f5-ba4cc5e1de8f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:35.534340 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9f135b-1aa2-4bd4-a3f5-ba4cc5e1de8f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:32:35.539237 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-7e9f135b-1aa2-4bd4-a3f5-ba4cc5e1de8f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:32:35.539237 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:32:35.539237 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:32:35.539237 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:32:35.539237 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:32:35.539237 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:32:35.539237 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:32:35.539237 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:32:35.539237 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:32:35.539237 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:32:35.539237 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:32:35.539237 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:32:35.539237 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:32:35.539237 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:32:35.539237 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:32:35.539237 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:32:35.540340 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:32:35.540340 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:32:35.540340 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:32:35.540340 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:32:35.540340 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:32:35.540340 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:32:35.540340 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:32:35.540340 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:32:35.540340 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:32:35.540340 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:32:35.540340 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:32:35.540340 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( Jul 28 17:32:35.540340 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:32:35.540340 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:32:35.540340 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:32:35.540340 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:32:35.541314 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:32:35.541314 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:32:35.541314 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:32:35.541314 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:32:35.541314 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:32:35.541314 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:32:35.541314 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:32:35.541314 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:32:35.541314 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:32:35.541314 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:32:35.541314 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:32:35.541314 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:32:35.541314 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 555/1087] 10.222.0.89 () {56 vars in 1096 bytes} [Thu Jul 28 17:32:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:32:35.547194 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbe18f9-1918-4762-b0d3-1e611263d318 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:35.573972 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:35.645741 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbe18f9-1918-4762-b0d3-1e611263d318 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:35.680381 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bccb06a-f760-4ac2-af94-96ec156c9e77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:35.681001 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bccb06a-f760-4ac2-af94-96ec156c9e77 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:35.681314 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bccb06a-f760-4ac2-af94-96ec156c9e77 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:35.681587 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bccb06a-f760-4ac2-af94-96ec156c9e77 None admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:35.682305 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bccb06a-f760-4ac2-af94-96ec156c9e77 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:32:35.682615 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bccb06a-f760-4ac2-af94-96ec156c9e77 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:35.683977 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bccb06a-f760-4ac2-af94-96ec156c9e77 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:35.692094 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 533/1088] 10.222.0.89 () {56 vars in 1099 bytes} [Thu Jul 28 17:32:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:32:35.698036 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9f135b-1aa2-4bd4-a3f5-ba4cc5e1de8f None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:35.741083 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:35.799260 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9f135b-1aa2-4bd4-a3f5-ba4cc5e1de8f None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:35.836827 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d744f134-4aec-426b-8064-4e0ab1471cf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:35.837502 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d744f134-4aec-426b-8064-4e0ab1471cf5 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:35.837777 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d744f134-4aec-426b-8064-4e0ab1471cf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:35.838044 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d744f134-4aec-426b-8064-4e0ab1471cf5 None admin] PATH_INFO: `/v3/users/1c079286ab384f4e86b21010b76f6fbf` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:35.839206 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d744f134-4aec-426b-8064-4e0ab1471cf5 None admin] RBAC: Authorizing `identity:get_user(user_id=1c079286ab384f4e86b21010b76f6fbf)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:35.840635 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d744f134-4aec-426b-8064-4e0ab1471cf5 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:35.843445 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 556/1089] 10.222.0.89 () {56 vars in 1177 bytes} [Thu Jul 28 17:32:35 2022] GET /identity/v3/users/1c079286ab384f4e86b21010b76f6fbf => generated 285 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:32:35.849438 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bccb06a-f760-4ac2-af94-96ec156c9e77 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:35.937664 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:35.949804 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bccb06a-f760-4ac2-af94-96ec156c9e77 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:35.984346 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba919a2-9539-4091-ac7e-0389d16d3e22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:35.984957 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba919a2-9539-4091-ac7e-0389d16d3e22 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:35.985264 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba919a2-9539-4091-ac7e-0389d16d3e22 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:35.985536 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba919a2-9539-4091-ac7e-0389d16d3e22 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:36.053398 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba919a2-9539-4091-ac7e-0389d16d3e22 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:36.054750 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba919a2-9539-4091-ac7e-0389d16d3e22 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:36.069038 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e464022-6cd4-4361-9f84-69cf9c3d6fd2 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:36.069441 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e464022-6cd4-4361-9f84-69cf9c3d6fd2 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:36.069811 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e464022-6cd4-4361-9f84-69cf9c3d6fd2 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:36.071180 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 557/1090] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:32:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 17:32:36.076639 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ef6511-1d68-408d-a210-dff207d17e73 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:36.076910 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ef6511-1d68-408d-a210-dff207d17e73 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:36.077331 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ef6511-1d68-408d-a210-dff207d17e73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:36.096681 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-a7ef6511-1d68-408d-a210-dff207d17e73 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:32:36.097936 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-a7ef6511-1d68-408d-a210-dff207d17e73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:32:36.142521 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:36.156484 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 534/1091] 10.222.0.89 () {58 vars in 1282 bytes} [Thu Jul 28 17:32:35 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 17:32:36.162464 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba919a2-9539-4091-ac7e-0389d16d3e22 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:36.166841 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 558/1092] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:32:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 17:32:36.190528 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ef6511-1d68-408d-a210-dff207d17e73 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:36.230424 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:36.263557 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba919a2-9539-4091-ac7e-0389d16d3e22 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:36.290958 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ef6511-1d68-408d-a210-dff207d17e73 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:36.298386 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95e6b95-5e4e-4a55-86fb-5b6bebe1f06d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:36.299456 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95e6b95-5e4e-4a55-86fb-5b6bebe1f06d None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:36.299742 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95e6b95-5e4e-4a55-86fb-5b6bebe1f06d None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:36.300017 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95e6b95-5e4e-4a55-86fb-5b6bebe1f06d None admin] PATH_INFO: `/v3/projects/9face3e074924844aff9ccb35b7ca307` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:36.301535 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95e6b95-5e4e-4a55-86fb-5b6bebe1f06d None admin] RBAC: Authorizing `identity:get_project(project_id=9face3e074924844aff9ccb35b7ca307)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:36.302966 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95e6b95-5e4e-4a55-86fb-5b6bebe1f06d None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:36.304966 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 535/1093] 10.222.0.89 () {56 vars in 1186 bytes} [Thu Jul 28 17:32:36 2022] GET /identity/v3/projects/9face3e074924844aff9ccb35b7ca307 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:32:36.311230 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95e6b95-5e4e-4a55-86fb-5b6bebe1f06d None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:36.325707 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fd496e-f96e-4f2e-8d5b-a4bc2b10f66c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:36.327543 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fd496e-f96e-4f2e-8d5b-a4bc2b10f66c None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:36.327815 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fd496e-f96e-4f2e-8d5b-a4bc2b10f66c None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:36.328088 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fd496e-f96e-4f2e-8d5b-a4bc2b10f66c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:36.338276 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fd496e-f96e-4f2e-8d5b-a4bc2b10f66c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:32:36.343666 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-37fd496e-f96e-4f2e-8d5b-a4bc2b10f66c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 17:32:36.343666 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:32:36.343666 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:32:36.343666 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:32:36.343666 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:32:36.343666 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:32:36.343666 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:32:36.343666 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:32:36.343666 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:32:36.343666 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:32:36.343666 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:32:36.343666 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:32:36.343666 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:32:36.343666 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:32:36.343666 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:32:36.343666 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:32:36.344821 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:32:36.344821 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:32:36.344821 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:32:36.344821 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:32:36.344821 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:32:36.344821 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:32:36.344821 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:32:36.344821 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:32:36.344821 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:32:36.344821 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:32:36.344821 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:32:36.344821 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( Jul 28 17:32:36.344821 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:32:36.344821 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:32:36.344821 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:32:36.344821 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:32:36.345745 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:32:36.345745 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:32:36.345745 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:32:36.345745 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:32:36.345745 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:32:36.345745 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:32:36.345745 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:32:36.345745 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:32:36.345745 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:32:36.345745 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:32:36.345745 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 17:32:36.345745 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:32:36.345745 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 559/1094] 10.222.0.89 () {56 vars in 1120 bytes} [Thu Jul 28 17:32:36 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:32:36.350940 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fd496e-f96e-4f2e-8d5b-a4bc2b10f66c None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:36.368325 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:36.413915 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95e6b95-5e4e-4a55-86fb-5b6bebe1f06d None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:36.446990 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:36.447161 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fd7235-cb33-4059-8b99-9b3c470dd2ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:36.447680 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fd7235-cb33-4059-8b99-9b3c470dd2ed None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:36.448016 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fd7235-cb33-4059-8b99-9b3c470dd2ed None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:36.448294 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fd7235-cb33-4059-8b99-9b3c470dd2ed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:36.449179 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fd7235-cb33-4059-8b99-9b3c470dd2ed None admin] RBAC: Adding query filter params (role.id=73e457231a824ea98c28d22610be640c, scope.project.id=9face3e074924844aff9ccb35b7ca307, user.id=1c079286ab384f4e86b21010b76f6fbf) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:32:36.449532 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fd7235-cb33-4059-8b99-9b3c470dd2ed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:36.451591 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fd7235-cb33-4059-8b99-9b3c470dd2ed None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:36.452140 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fd496e-f96e-4f2e-8d5b-a4bc2b10f66c None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:36.457973 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 536/1095] 10.222.0.89 () {56 vars in 1374 bytes} [Thu Jul 28 17:32:36 2022] GET /identity/v3/role_assignments?user.id=1c079286ab384f4e86b21010b76f6fbf&scope.project.id=9face3e074924844aff9ccb35b7ca307&role.id=73e457231a824ea98c28d22610be640c => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:32:36.487606 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f758706-169c-4817-92b2-9126c64b4f19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:36.488243 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f758706-169c-4817-92b2-9126c64b4f19 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:36.488527 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f758706-169c-4817-92b2-9126c64b4f19 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:36.488801 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f758706-169c-4817-92b2-9126c64b4f19 None admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:36.489568 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f758706-169c-4817-92b2-9126c64b4f19 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:32:36.489878 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f758706-169c-4817-92b2-9126c64b4f19 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:36.491283 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f758706-169c-4817-92b2-9126c64b4f19 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:36.502954 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 560/1096] 10.222.0.89 () {56 vars in 1115 bytes} [Thu Jul 28 17:32:36 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:32:36.508786 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fd7235-cb33-4059-8b99-9b3c470dd2ed None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:36.616583 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fd7235-cb33-4059-8b99-9b3c470dd2ed None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:36.637101 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:36.652445 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa47a9a-a59b-4f40-8cd8-973b119db737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:36.653091 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa47a9a-a59b-4f40-8cd8-973b119db737 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:36.653431 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa47a9a-a59b-4f40-8cd8-973b119db737 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:36.653710 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa47a9a-a59b-4f40-8cd8-973b119db737 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:36.664424 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa47a9a-a59b-4f40-8cd8-973b119db737 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:36.665893 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa47a9a-a59b-4f40-8cd8-973b119db737 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:36.681329 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-efa47a9a-a59b-4f40-8cd8-973b119db737 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:32:36.681329 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:32:36.681329 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:32:36.681329 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:32:36.681329 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:32:36.681329 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:32:36.681329 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:32:36.681329 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:32:36.681329 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:32:36.681329 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:32:36.681329 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:32:36.681329 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:32:36.681329 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:32:36.681329 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:32:36.681329 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:32:36.682440 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:32:36.682440 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:32:36.682440 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:32:36.682440 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:32:36.682440 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:32:36.682440 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:32:36.682440 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:32:36.682440 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:32:36.682440 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:32:36.682440 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:32:36.682440 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:32:36.682440 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:32:36.682440 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:32:36.682440 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:32:36.682440 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:32:36.682440 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:32:36.682440 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:32:36.683559 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:32:36.683559 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:32:36.683559 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:32:36.683559 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:32:36.684093 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 537/1097] 10.222.0.89 () {56 vars in 1108 bytes} [Thu Jul 28 17:32:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:32:36.689602 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f758706-169c-4817-92b2-9126c64b4f19 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:36.790474 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f758706-169c-4817-92b2-9126c64b4f19 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:36.825190 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f83e229-8268-4ab3-a6a2-0d096eaa4682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:36.825889 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f83e229-8268-4ab3-a6a2-0d096eaa4682 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:36.826666 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f83e229-8268-4ab3-a6a2-0d096eaa4682 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:36.827337 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f83e229-8268-4ab3-a6a2-0d096eaa4682 None admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:36.827969 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f83e229-8268-4ab3-a6a2-0d096eaa4682 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:32:36.828284 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f83e229-8268-4ab3-a6a2-0d096eaa4682 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:36.829704 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f83e229-8268-4ab3-a6a2-0d096eaa4682 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:36.841759 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 561/1098] 10.222.0.89 () {56 vars in 1109 bytes} [Thu Jul 28 17:32:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:32:36.848269 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa47a9a-a59b-4f40-8cd8-973b119db737 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:36.949779 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa47a9a-a59b-4f40-8cd8-973b119db737 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:36.983390 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c88a83-5696-4217-a98e-220007e8a2bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:36.984026 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c88a83-5696-4217-a98e-220007e8a2bc None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:36.984306 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c88a83-5696-4217-a98e-220007e8a2bc None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:36.984578 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c88a83-5696-4217-a98e-220007e8a2bc None admin] PATH_INFO: `/v3/users/nova` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:36.992128 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c88a83-5696-4217-a98e-220007e8a2bc None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:36.993598 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c88a83-5696-4217-a98e-220007e8a2bc None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:37.005329 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-83c88a83-5696-4217-a98e-220007e8a2bc None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 17:32:37.005329 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:32:37.005329 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:32:37.005329 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:32:37.005329 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:32:37.005329 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:32:37.005329 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:32:37.005329 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:32:37.005329 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:32:37.005329 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:32:37.005329 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:32:37.005329 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:32:37.005329 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 17:32:37.005329 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 17:32:37.005329 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 17:32:37.005329 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 17:32:37.006541 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:32:37.006541 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:32:37.006541 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 17:32:37.006541 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:32:37.006541 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 17:32:37.006541 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:32:37.006541 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:32:37.006541 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:32:37.006541 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:32:37.006541 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:32:37.006541 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:32:37.006541 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:32:37.006541 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:32:37.006541 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:32:37.006541 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:32:37.006541 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:32:37.006541 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:32:37.006541 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:32:37.007497 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:32:37.007497 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:32:37.007497 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 17:32:37.007497 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 17:32:37.007497 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 17:32:37.007497 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 17:32:37.007497 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 17:32:37.007497 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 17:32:37.007497 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 17:32:37.007497 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:32:37.007497 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 538/1099] 10.222.0.89 () {56 vars in 1093 bytes} [Thu Jul 28 17:32:36 2022] GET /identity/v3/users/nova => generated 82 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:32:37.012346 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f83e229-8268-4ab3-a6a2-0d096eaa4682 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:37.035175 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:37.114491 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f83e229-8268-4ab3-a6a2-0d096eaa4682 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:37.151041 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2ce2a5-0617-4698-9a8c-568d886aea9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:37.151651 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2ce2a5-0617-4698-9a8c-568d886aea9c None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:37.151936 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2ce2a5-0617-4698-9a8c-568d886aea9c None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:37.152209 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2ce2a5-0617-4698-9a8c-568d886aea9c None admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:37.152995 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2ce2a5-0617-4698-9a8c-568d886aea9c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:32:37.153339 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2ce2a5-0617-4698-9a8c-568d886aea9c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:37.154962 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2ce2a5-0617-4698-9a8c-568d886aea9c None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:37.171774 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 562/1100] 10.222.0.89 () {56 vars in 1133 bytes} [Thu Jul 28 17:32:37 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:32:37.177559 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c88a83-5696-4217-a98e-220007e8a2bc None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:37.209820 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:37.277885 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c88a83-5696-4217-a98e-220007e8a2bc None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:37.311245 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10e9897-d397-45ec-895a-0dbd5a59e66a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:37.311875 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10e9897-d397-45ec-895a-0dbd5a59e66a None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:37.312138 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10e9897-d397-45ec-895a-0dbd5a59e66a None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:37.312412 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10e9897-d397-45ec-895a-0dbd5a59e66a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:37.378139 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10e9897-d397-45ec-895a-0dbd5a59e66a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:37.379570 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10e9897-d397-45ec-895a-0dbd5a59e66a None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:37.408764 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:37.477043 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 539/1101] 10.222.0.89 () {58 vars in 1282 bytes} [Thu Jul 28 17:32:37 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 17:32:37.483214 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2ce2a5-0617-4698-9a8c-568d886aea9c None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:37.563766 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:37.585542 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2ce2a5-0617-4698-9a8c-568d886aea9c None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:37.624911 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a3b63e-5ca6-4a3b-83f2-7e96c8548b65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:37.625610 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a3b63e-5ca6-4a3b-83f2-7e96c8548b65 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:37.625886 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a3b63e-5ca6-4a3b-83f2-7e96c8548b65 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:37.626155 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a3b63e-5ca6-4a3b-83f2-7e96c8548b65 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:37.635996 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a3b63e-5ca6-4a3b-83f2-7e96c8548b65 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:37.637459 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a3b63e-5ca6-4a3b-83f2-7e96c8548b65 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:37.652244 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-50a3b63e-5ca6-4a3b-83f2-7e96c8548b65 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:32:37.652244 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:32:37.652244 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:32:37.652244 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:32:37.652244 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:32:37.652244 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:32:37.652244 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:32:37.652244 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:32:37.652244 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:32:37.652244 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:32:37.652244 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:32:37.652244 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:32:37.652244 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:32:37.652244 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:32:37.652244 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:32:37.653132 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:32:37.653132 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:32:37.653132 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:32:37.653132 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:32:37.653132 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:32:37.653132 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:32:37.653132 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:32:37.653132 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:32:37.653132 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( Jul 28 17:32:37.653132 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:32:37.653132 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:32:37.653132 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:32:37.653132 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:32:37.653132 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:32:37.653132 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:32:37.653132 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:32:37.653132 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:32:37.654778 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:32:37.654778 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:32:37.654778 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:32:37.654778 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:32:37.654778 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 563/1102] 10.222.0.89 () {56 vars in 1108 bytes} [Thu Jul 28 17:32:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:32:37.660038 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10e9897-d397-45ec-895a-0dbd5a59e66a None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:37.756907 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10e9897-d397-45ec-895a-0dbd5a59e66a None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:37.785894 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:37.791195 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb1ed88-7ee9-4cc3-9b1d-a4918e4e5639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:37.791805 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb1ed88-7ee9-4cc3-9b1d-a4918e4e5639 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:37.792084 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb1ed88-7ee9-4cc3-9b1d-a4918e4e5639 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:37.792354 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb1ed88-7ee9-4cc3-9b1d-a4918e4e5639 None admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:37.793046 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb1ed88-7ee9-4cc3-9b1d-a4918e4e5639 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:32:37.793380 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb1ed88-7ee9-4cc3-9b1d-a4918e4e5639 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:37.794750 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb1ed88-7ee9-4cc3-9b1d-a4918e4e5639 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:37.806265 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 540/1103] 10.222.0.89 () {56 vars in 1109 bytes} [Thu Jul 28 17:32:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:32:37.812300 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a3b63e-5ca6-4a3b-83f2-7e96c8548b65 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:37.914042 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a3b63e-5ca6-4a3b-83f2-7e96c8548b65 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:37.942594 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:37.949131 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa325f9-f3cf-439f-8b65-beb8a151f7ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:37.949798 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa325f9-f3cf-439f-8b65-beb8a151f7ca None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:37.950080 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa325f9-f3cf-439f-8b65-beb8a151f7ca None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:37.950353 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa325f9-f3cf-439f-8b65-beb8a151f7ca None admin] PATH_INFO: `/v3/projects/service` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:37.957041 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa325f9-f3cf-439f-8b65-beb8a151f7ca None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:37.958507 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa325f9-f3cf-439f-8b65-beb8a151f7ca None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:37.970142 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-9fa325f9-f3cf-439f-8b65-beb8a151f7ca None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:32:37.970142 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:32:37.970142 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:32:37.970142 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:32:37.970142 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:32:37.970142 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:32:37.970142 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:32:37.970142 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:32:37.970142 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:32:37.970142 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:32:37.970142 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:32:37.970142 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:32:37.970142 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 17:32:37.970142 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 17:32:37.970142 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 17:32:37.971228 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 17:32:37.971228 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:32:37.971228 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:32:37.971228 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:32:37.971228 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:32:37.971228 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:32:37.971228 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:32:37.971228 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:32:37.971228 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( Jul 28 17:32:37.971228 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:32:37.971228 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:32:37.971228 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:32:37.971228 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:32:37.971228 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:32:37.971228 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:32:37.971228 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:32:37.971228 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:32:37.972149 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 17:32:37.972149 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 17:32:37.972149 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 17:32:37.972149 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 17:32:37.972149 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 17:32:37.972149 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 17:32:37.972149 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:32:37.972149 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:32:37.972149 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 564/1104] 10.222.0.89 () {56 vars in 1111 bytes} [Thu Jul 28 17:32:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:32:37.977086 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb1ed88-7ee9-4cc3-9b1d-a4918e4e5639 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:38.076325 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb1ed88-7ee9-4cc3-9b1d-a4918e4e5639 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:38.112318 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c182967e-74f6-4952-9fa1-6f6c53331725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:38.112948 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c182967e-74f6-4952-9fa1-6f6c53331725 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:38.113263 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c182967e-74f6-4952-9fa1-6f6c53331725 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:38.113543 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c182967e-74f6-4952-9fa1-6f6c53331725 None admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:38.114298 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c182967e-74f6-4952-9fa1-6f6c53331725 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:32:38.114607 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c182967e-74f6-4952-9fa1-6f6c53331725 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:38.116030 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c182967e-74f6-4952-9fa1-6f6c53331725 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:38.129197 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 541/1105] 10.222.0.89 () {56 vars in 1148 bytes} [Thu Jul 28 17:32:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:32:38.135427 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa325f9-f3cf-439f-8b65-beb8a151f7ca None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:38.235099 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa325f9-f3cf-439f-8b65-beb8a151f7ca None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:38.268218 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e0f345-1b9b-40b6-a214-7212dd076c3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:38.268830 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e0f345-1b9b-40b6-a214-7212dd076c3c None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:38.269111 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e0f345-1b9b-40b6-a214-7212dd076c3c None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:38.269410 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e0f345-1b9b-40b6-a214-7212dd076c3c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:38.270188 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e0f345-1b9b-40b6-a214-7212dd076c3c None admin] RBAC: Adding query filter params (role.id=af4b58a9c88d4cb4861277524ab3e1ed, scope.project.id=04434e8535a64044996316f1d17dc0bc, user.id=d10b49b29df94404a71f43273839c252) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:32:38.270494 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e0f345-1b9b-40b6-a214-7212dd076c3c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:38.271923 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e0f345-1b9b-40b6-a214-7212dd076c3c None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:38.277869 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 565/1106] 10.222.0.89 () {56 vars in 1374 bytes} [Thu Jul 28 17:32:38 2022] GET /identity/v3/role_assignments?user.id=d10b49b29df94404a71f43273839c252&scope.project.id=04434e8535a64044996316f1d17dc0bc&role.id=af4b58a9c88d4cb4861277524ab3e1ed => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:32:38.514415 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ef300d-738f-43c5-bf50-70900aea15fa None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:38.514964 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ef300d-738f-43c5-bf50-70900aea15fa None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:38.515061 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ef300d-738f-43c5-bf50-70900aea15fa None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:38.516418 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 542/1107] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:32:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:32:38.521762 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b1e697-8edc-4d83-abaf-57fd01d30133 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:38.522170 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b1e697-8edc-4d83-abaf-57fd01d30133 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:38.522365 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b1e697-8edc-4d83-abaf-57fd01d30133 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:38.540447 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-32b1e697-8edc-4d83-abaf-57fd01d30133 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:32:38.541688 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-32b1e697-8edc-4d83-abaf-57fd01d30133 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:32:38.597262 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:38.615825 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 566/1108] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:32:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:32:38.642472 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ef300d-738f-43c5-bf50-70900aea15fa None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:38.700007 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:38.750864 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ef300d-738f-43c5-bf50-70900aea15fa None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:38.786666 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b90062-fae5-4723-9fb4-8a3c8c934402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:38.787242 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b90062-fae5-4723-9fb4-8a3c8c934402 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:38.787527 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b90062-fae5-4723-9fb4-8a3c8c934402 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:38.787801 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b90062-fae5-4723-9fb4-8a3c8c934402 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:38.788975 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b90062-fae5-4723-9fb4-8a3c8c934402 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:38.790433 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b90062-fae5-4723-9fb4-8a3c8c934402 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:38.792056 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 543/1109] 10.222.0.89 () {56 vars in 1108 bytes} [Thu Jul 28 17:32:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:32:38.798859 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b1e697-8edc-4d83-abaf-57fd01d30133 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:38.903240 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b1e697-8edc-4d83-abaf-57fd01d30133 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:38.940455 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e21e9b4-e596-4fcf-91f0-837b6a4af38d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:38.941081 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e21e9b4-e596-4fcf-91f0-837b6a4af38d None admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:38.941393 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e21e9b4-e596-4fcf-91f0-837b6a4af38d None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:38.941670 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e21e9b4-e596-4fcf-91f0-837b6a4af38d None admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:38.942369 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e21e9b4-e596-4fcf-91f0-837b6a4af38d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:38.943804 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e21e9b4-e596-4fcf-91f0-837b6a4af38d None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:38.966019 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 567/1110] 10.222.0.89 () {60 vars in 1132 bytes} [Thu Jul 28 17:32:38 2022] POST /identity/v3/users => generated 286 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:32:41.028512 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78457a3-ab8d-46d6-9a47-4196625b0347 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:41.029120 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78457a3-ab8d-46d6-9a47-4196625b0347 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:41.029120 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78457a3-ab8d-46d6-9a47-4196625b0347 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:41.030786 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 544/1111] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:32:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:32:41.036247 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b711c18e-6b42-4880-adfe-e29782b9fade None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:41.036715 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b711c18e-6b42-4880-adfe-e29782b9fade None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:41.036815 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b711c18e-6b42-4880-adfe-e29782b9fade None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:41.055367 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-b711c18e-6b42-4880-adfe-e29782b9fade None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:32:41.056565 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-b711c18e-6b42-4880-adfe-e29782b9fade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:32:41.090145 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:41.125035 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 568/1112] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:32:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:32:41.148050 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78457a3-ab8d-46d6-9a47-4196625b0347 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:41.246340 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78457a3-ab8d-46d6-9a47-4196625b0347 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:41.266585 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:41.281107 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8174842b-b3b8-485c-a45d-d37d262d0986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:41.281777 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8174842b-b3b8-485c-a45d-d37d262d0986 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:41.282060 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8174842b-b3b8-485c-a45d-d37d262d0986 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:41.282338 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8174842b-b3b8-485c-a45d-d37d262d0986 None admin] PATH_INFO: `/v3/roles/895fd016dffb41e7b0205c9a55a4e0f8` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:41.283790 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8174842b-b3b8-485c-a45d-d37d262d0986 None admin] RBAC: Authorizing `identity:get_role(role_id=895fd016dffb41e7b0205c9a55a4e0f8)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:41.285147 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8174842b-b3b8-485c-a45d-d37d262d0986 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:41.286669 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 545/1113] 10.222.0.89 () {56 vars in 1177 bytes} [Thu Jul 28 17:32:41 2022] GET /identity/v3/roles/895fd016dffb41e7b0205c9a55a4e0f8 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:32:41.292445 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b711c18e-6b42-4880-adfe-e29782b9fade None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:41.387747 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b711c18e-6b42-4880-adfe-e29782b9fade None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:41.421399 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5eed00-6603-453b-9445-db1c9ae45134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:41.422023 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5eed00-6603-453b-9445-db1c9ae45134 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:41.422303 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5eed00-6603-453b-9445-db1c9ae45134 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:41.422578 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5eed00-6603-453b-9445-db1c9ae45134 None admin] PATH_INFO: `/v3/users/6734d1a8c57344b1a301666fb8f1b924` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:41.435671 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5eed00-6603-453b-9445-db1c9ae45134 None admin] RBAC: Authorizing `identity:get_user(user_id=6734d1a8c57344b1a301666fb8f1b924)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:41.437031 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5eed00-6603-453b-9445-db1c9ae45134 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:41.438708 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 569/1114] 10.222.0.89 () {56 vars in 1177 bytes} [Thu Jul 28 17:32:41 2022] GET /identity/v3/users/6734d1a8c57344b1a301666fb8f1b924 => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:32:41.444454 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8174842b-b3b8-485c-a45d-d37d262d0986 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:41.544152 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8174842b-b3b8-485c-a45d-d37d262d0986 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:41.578009 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964c3e14-9e89-4a2f-9b67-f3e6f8e70d31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:41.578617 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964c3e14-9e89-4a2f-9b67-f3e6f8e70d31 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:41.578895 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964c3e14-9e89-4a2f-9b67-f3e6f8e70d31 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:41.579164 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964c3e14-9e89-4a2f-9b67-f3e6f8e70d31 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:41.633993 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:41.646120 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964c3e14-9e89-4a2f-9b67-f3e6f8e70d31 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:41.647533 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964c3e14-9e89-4a2f-9b67-f3e6f8e70d31 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:41.743025 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 546/1115] 10.222.0.89 () {58 vars in 1282 bytes} [Thu Jul 28 17:32:41 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 17:32:41.749291 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5eed00-6603-453b-9445-db1c9ae45134 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:41.776763 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:41.848257 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5eed00-6603-453b-9445-db1c9ae45134 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:41.882756 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ee9c4b-393e-4b10-93d6-a9c20d3741a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:41.883404 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ee9c4b-393e-4b10-93d6-a9c20d3741a0 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:41.883687 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ee9c4b-393e-4b10-93d6-a9c20d3741a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:41.883965 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ee9c4b-393e-4b10-93d6-a9c20d3741a0 None admin] PATH_INFO: `/v3/projects/9face3e074924844aff9ccb35b7ca307` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:41.885138 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ee9c4b-393e-4b10-93d6-a9c20d3741a0 None admin] RBAC: Authorizing `identity:get_project(project_id=9face3e074924844aff9ccb35b7ca307)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:41.886715 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ee9c4b-393e-4b10-93d6-a9c20d3741a0 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:41.888392 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 570/1116] 10.222.0.89 () {56 vars in 1186 bytes} [Thu Jul 28 17:32:41 2022] GET /identity/v3/projects/9face3e074924844aff9ccb35b7ca307 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:32:41.895315 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964c3e14-9e89-4a2f-9b67-f3e6f8e70d31 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:41.979869 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:41.993613 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964c3e14-9e89-4a2f-9b67-f3e6f8e70d31 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:42.026473 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9bbebd-25c6-4e5d-8a6a-4d4b8660f89e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:42.027077 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9bbebd-25c6-4e5d-8a6a-4d4b8660f89e None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:42.027357 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9bbebd-25c6-4e5d-8a6a-4d4b8660f89e None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:42.027628 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9bbebd-25c6-4e5d-8a6a-4d4b8660f89e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:42.028413 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9bbebd-25c6-4e5d-8a6a-4d4b8660f89e None admin] RBAC: Adding query filter params (role.id=895fd016dffb41e7b0205c9a55a4e0f8, scope.project.id=9face3e074924844aff9ccb35b7ca307, user.id=6734d1a8c57344b1a301666fb8f1b924) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:32:42.028719 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9bbebd-25c6-4e5d-8a6a-4d4b8660f89e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:42.031028 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9bbebd-25c6-4e5d-8a6a-4d4b8660f89e None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:42.036289 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 547/1117] 10.222.0.89 () {56 vars in 1374 bytes} [Thu Jul 28 17:32:41 2022] GET /identity/v3/role_assignments?user.id=6734d1a8c57344b1a301666fb8f1b924&scope.project.id=9face3e074924844aff9ccb35b7ca307&role.id=895fd016dffb41e7b0205c9a55a4e0f8 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:32:44.009552 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462005dc-cc21-43a5-a8c2-98a3a22014c9 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:44.010113 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462005dc-cc21-43a5-a8c2-98a3a22014c9 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:44.010113 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462005dc-cc21-43a5-a8c2-98a3a22014c9 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:44.011429 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 571/1118] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:32:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:32:44.017381 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76410d2-0a0e-4281-a588-cdc6306c4beb None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:44.017789 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76410d2-0a0e-4281-a588-cdc6306c4beb None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:44.017922 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76410d2-0a0e-4281-a588-cdc6306c4beb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:44.035891 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-d76410d2-0a0e-4281-a588-cdc6306c4beb None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:32:44.037079 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-d76410d2-0a0e-4281-a588-cdc6306c4beb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:32:44.101606 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 548/1119] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:32:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:32:44.124742 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462005dc-cc21-43a5-a8c2-98a3a22014c9 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:44.218099 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:44.224796 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462005dc-cc21-43a5-a8c2-98a3a22014c9 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:44.258989 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d47d76c-5689-4047-acf2-16bb67d4d613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:44.259602 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d47d76c-5689-4047-acf2-16bb67d4d613 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:44.259878 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d47d76c-5689-4047-acf2-16bb67d4d613 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:44.260147 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d47d76c-5689-4047-acf2-16bb67d4d613 None admin] PATH_INFO: `/v3/roles/895fd016dffb41e7b0205c9a55a4e0f8` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:44.261593 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d47d76c-5689-4047-acf2-16bb67d4d613 None admin] RBAC: Authorizing `identity:get_role(role_id=895fd016dffb41e7b0205c9a55a4e0f8)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:44.262949 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d47d76c-5689-4047-acf2-16bb67d4d613 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:44.264683 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 572/1120] 10.222.0.89 () {56 vars in 1177 bytes} [Thu Jul 28 17:32:44 2022] GET /identity/v3/roles/895fd016dffb41e7b0205c9a55a4e0f8 => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:32:44.270560 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76410d2-0a0e-4281-a588-cdc6306c4beb None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:44.349598 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:44.368706 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76410d2-0a0e-4281-a588-cdc6306c4beb None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:44.401990 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b8e975-9016-4f95-b0fe-44303a1c4dbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:44.402622 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b8e975-9016-4f95-b0fe-44303a1c4dbf None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:44.402900 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b8e975-9016-4f95-b0fe-44303a1c4dbf None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:44.403170 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b8e975-9016-4f95-b0fe-44303a1c4dbf None admin] PATH_INFO: `/v3/users/6734d1a8c57344b1a301666fb8f1b924` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:44.405076 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b8e975-9016-4f95-b0fe-44303a1c4dbf None admin] RBAC: Authorizing `identity:get_user(user_id=6734d1a8c57344b1a301666fb8f1b924)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:44.406270 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b8e975-9016-4f95-b0fe-44303a1c4dbf None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:44.407867 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 549/1121] 10.222.0.89 () {56 vars in 1177 bytes} [Thu Jul 28 17:32:44 2022] GET /identity/v3/users/6734d1a8c57344b1a301666fb8f1b924 => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:32:44.413814 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d47d76c-5689-4047-acf2-16bb67d4d613 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:44.510878 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d47d76c-5689-4047-acf2-16bb67d4d613 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:44.544692 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380ba2e4-90a4-4a3c-b7a1-1b51f24d5a35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:44.545946 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380ba2e4-90a4-4a3c-b7a1-1b51f24d5a35 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:44.546178 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380ba2e4-90a4-4a3c-b7a1-1b51f24d5a35 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:44.546454 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380ba2e4-90a4-4a3c-b7a1-1b51f24d5a35 None admin] PATH_INFO: `/v3/projects/9face3e074924844aff9ccb35b7ca307` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:44.547613 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380ba2e4-90a4-4a3c-b7a1-1b51f24d5a35 None admin] RBAC: Authorizing `identity:get_project(project_id=9face3e074924844aff9ccb35b7ca307)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:44.549041 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380ba2e4-90a4-4a3c-b7a1-1b51f24d5a35 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:44.550718 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 573/1122] 10.222.0.89 () {56 vars in 1186 bytes} [Thu Jul 28 17:32:44 2022] GET /identity/v3/projects/9face3e074924844aff9ccb35b7ca307 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:32:44.556579 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b8e975-9016-4f95-b0fe-44303a1c4dbf None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:44.653657 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b8e975-9016-4f95-b0fe-44303a1c4dbf None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:44.690207 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040fba65-bbfa-4f66-8ff5-7299d23810d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:44.690861 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040fba65-bbfa-4f66-8ff5-7299d23810d4 None admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:44.691135 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040fba65-bbfa-4f66-8ff5-7299d23810d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:44.691402 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040fba65-bbfa-4f66-8ff5-7299d23810d4 None admin] PATH_INFO: `/v3/projects/9face3e074924844aff9ccb35b7ca307/users/6734d1a8c57344b1a301666fb8f1b924/roles/895fd016dffb41e7b0205c9a55a4e0f8` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:44.693541 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040fba65-bbfa-4f66-8ff5-7299d23810d4 None admin] RBAC: Authorizing `identity:create_grant(project_id=9face3e074924844aff9ccb35b7ca307, user_id=6734d1a8c57344b1a301666fb8f1b924, role_id=895fd016dffb41e7b0205c9a55a4e0f8)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:44.694958 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040fba65-bbfa-4f66-8ff5-7299d23810d4 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:44.708071 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 550/1123] 10.222.0.89 () {58 vars in 1439 bytes} [Thu Jul 28 17:32:44 2022] PUT /identity/v3/projects/9face3e074924844aff9ccb35b7ca307/users/6734d1a8c57344b1a301666fb8f1b924/roles/895fd016dffb41e7b0205c9a55a4e0f8 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:32:46.717477 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914a49db-10cf-4e0a-a1b6-afb79a71eba2 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:46.718036 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914a49db-10cf-4e0a-a1b6-afb79a71eba2 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:46.718036 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914a49db-10cf-4e0a-a1b6-afb79a71eba2 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:46.719371 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 574/1124] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:32:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:32:46.724699 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49244d34-9d24-4a22-8abc-7adc21b34899 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:46.725087 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49244d34-9d24-4a22-8abc-7adc21b34899 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:46.725280 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49244d34-9d24-4a22-8abc-7adc21b34899 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:46.742956 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-49244d34-9d24-4a22-8abc-7adc21b34899 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:32:46.744139 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-49244d34-9d24-4a22-8abc-7adc21b34899 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:32:46.747142 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:46.809750 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 551/1125] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:32:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:32:46.833747 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914a49db-10cf-4e0a-a1b6-afb79a71eba2 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:46.864895 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:46.934817 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914a49db-10cf-4e0a-a1b6-afb79a71eba2 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:46.968467 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634d6619-617a-493d-807d-d80070614553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:46.969025 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634d6619-617a-493d-807d-d80070614553 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:46.969332 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634d6619-617a-493d-807d-d80070614553 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:46.969607 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634d6619-617a-493d-807d-d80070614553 None admin] PATH_INFO: `/v3/roles/895fd016dffb41e7b0205c9a55a4e0f8` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:46.971044 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634d6619-617a-493d-807d-d80070614553 None admin] RBAC: Authorizing `identity:get_role(role_id=895fd016dffb41e7b0205c9a55a4e0f8)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:46.972399 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634d6619-617a-493d-807d-d80070614553 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:46.973754 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 575/1126] 10.222.0.89 () {56 vars in 1177 bytes} [Thu Jul 28 17:32:46 2022] GET /identity/v3/roles/895fd016dffb41e7b0205c9a55a4e0f8 => generated 223 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:32:46.979679 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49244d34-9d24-4a22-8abc-7adc21b34899 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:47.078238 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49244d34-9d24-4a22-8abc-7adc21b34899 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:47.111437 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070cc488-47fb-4772-9580-5152b7bc35bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:47.112056 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070cc488-47fb-4772-9580-5152b7bc35bc None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:47.112330 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070cc488-47fb-4772-9580-5152b7bc35bc None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:47.112598 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070cc488-47fb-4772-9580-5152b7bc35bc None admin] PATH_INFO: `/v3/users/6734d1a8c57344b1a301666fb8f1b924` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:47.113757 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070cc488-47fb-4772-9580-5152b7bc35bc None admin] RBAC: Authorizing `identity:get_user(user_id=6734d1a8c57344b1a301666fb8f1b924)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:47.115174 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070cc488-47fb-4772-9580-5152b7bc35bc None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:47.116754 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 552/1127] 10.222.0.89 () {56 vars in 1177 bytes} [Thu Jul 28 17:32:46 2022] GET /identity/v3/users/6734d1a8c57344b1a301666fb8f1b924 => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:32:47.122778 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634d6619-617a-493d-807d-d80070614553 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:47.217007 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:47.220473 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634d6619-617a-493d-807d-d80070614553 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:47.254752 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20a0e47-cd13-475e-b234-a7b4e77c211d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:47.255353 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20a0e47-cd13-475e-b234-a7b4e77c211d None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:47.255633 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20a0e47-cd13-475e-b234-a7b4e77c211d None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:47.255908 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20a0e47-cd13-475e-b234-a7b4e77c211d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:47.321022 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20a0e47-cd13-475e-b234-a7b4e77c211d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:47.322502 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20a0e47-cd13-475e-b234-a7b4e77c211d None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:47.414222 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:47.422282 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 576/1128] 10.222.0.89 () {58 vars in 1282 bytes} [Thu Jul 28 17:32:47 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 17:32:47.424301 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070cc488-47fb-4772-9580-5152b7bc35bc None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:47.434801 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:47.522732 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070cc488-47fb-4772-9580-5152b7bc35bc None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:47.556411 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9228610-5b3b-4dc5-8678-0dbc458557f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:47.557066 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9228610-5b3b-4dc5-8678-0dbc458557f1 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:47.557371 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9228610-5b3b-4dc5-8678-0dbc458557f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:47.557645 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9228610-5b3b-4dc5-8678-0dbc458557f1 None admin] PATH_INFO: `/v3/projects/9face3e074924844aff9ccb35b7ca307` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:47.558801 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9228610-5b3b-4dc5-8678-0dbc458557f1 None admin] RBAC: Authorizing `identity:get_project(project_id=9face3e074924844aff9ccb35b7ca307)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:47.560222 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9228610-5b3b-4dc5-8678-0dbc458557f1 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:47.562598 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 553/1129] 10.222.0.89 () {56 vars in 1186 bytes} [Thu Jul 28 17:32:47 2022] GET /identity/v3/projects/9face3e074924844aff9ccb35b7ca307 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:32:47.568704 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20a0e47-cd13-475e-b234-a7b4e77c211d None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:47.664402 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20a0e47-cd13-475e-b234-a7b4e77c211d None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:47.697038 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427e1b65-490e-4593-8e56-d35e15a73947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:47.698442 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427e1b65-490e-4593-8e56-d35e15a73947 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:47.698442 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427e1b65-490e-4593-8e56-d35e15a73947 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:47.698442 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427e1b65-490e-4593-8e56-d35e15a73947 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:47.698973 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427e1b65-490e-4593-8e56-d35e15a73947 None admin] RBAC: Adding query filter params (role.id=895fd016dffb41e7b0205c9a55a4e0f8, scope.project.id=9face3e074924844aff9ccb35b7ca307, user.id=6734d1a8c57344b1a301666fb8f1b924) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:32:47.699316 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427e1b65-490e-4593-8e56-d35e15a73947 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:47.700679 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427e1b65-490e-4593-8e56-d35e15a73947 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:47.706499 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 577/1130] 10.222.0.89 () {56 vars in 1374 bytes} [Thu Jul 28 17:32:47 2022] GET /identity/v3/role_assignments?user.id=6734d1a8c57344b1a301666fb8f1b924&scope.project.id=9face3e074924844aff9ccb35b7ca307&role.id=895fd016dffb41e7b0205c9a55a4e0f8 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:32:49.740938 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581842ad-aa71-4b09-9de0-510a3a41aed5 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:49.741541 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581842ad-aa71-4b09-9de0-510a3a41aed5 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:49.741650 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581842ad-aa71-4b09-9de0-510a3a41aed5 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:49.743013 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 554/1131] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:32:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:32:49.748860 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e103dc1-f1a5-45cf-b8e2-ec0246b80968 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:49.749279 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e103dc1-f1a5-45cf-b8e2-ec0246b80968 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:49.749431 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e103dc1-f1a5-45cf-b8e2-ec0246b80968 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:49.767641 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-4e103dc1-f1a5-45cf-b8e2-ec0246b80968 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:32:49.768843 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-4e103dc1-f1a5-45cf-b8e2-ec0246b80968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:32:49.825524 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:49.835966 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 578/1132] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:32:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:32:49.858951 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581842ad-aa71-4b09-9de0-510a3a41aed5 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:49.939050 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:49.962639 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581842ad-aa71-4b09-9de0-510a3a41aed5 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:49.994979 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247f923e-f7fc-42f9-b3d8-1e0fc714555a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:49.995535 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247f923e-f7fc-42f9-b3d8-1e0fc714555a None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:49.995813 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247f923e-f7fc-42f9-b3d8-1e0fc714555a None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:49.996086 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247f923e-f7fc-42f9-b3d8-1e0fc714555a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:49.997524 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247f923e-f7fc-42f9-b3d8-1e0fc714555a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:49.999501 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247f923e-f7fc-42f9-b3d8-1e0fc714555a None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:50.001099 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 555/1133] 10.222.0.89 () {56 vars in 1108 bytes} [Thu Jul 28 17:32:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:32:50.007100 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e103dc1-f1a5-45cf-b8e2-ec0246b80968 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:50.105528 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e103dc1-f1a5-45cf-b8e2-ec0246b80968 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:50.139649 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b763672-035d-430b-953f-aeec69e68a1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:50.140260 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b763672-035d-430b-953f-aeec69e68a1a None admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:50.140538 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b763672-035d-430b-953f-aeec69e68a1a None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:50.140805 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b763672-035d-430b-953f-aeec69e68a1a None admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:50.141500 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b763672-035d-430b-953f-aeec69e68a1a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:50.142914 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b763672-035d-430b-953f-aeec69e68a1a None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:50.167917 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 579/1134] 10.222.0.89 () {60 vars in 1141 bytes} [Thu Jul 28 17:32:50 2022] POST /identity/v3/projects => generated 311 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:32:52.192594 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b889d918-c30c-4e43-87cb-23f940f3d8ed None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:52.193133 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b889d918-c30c-4e43-87cb-23f940f3d8ed None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:52.193133 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b889d918-c30c-4e43-87cb-23f940f3d8ed None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:52.194531 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 556/1135] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:32:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:32:52.199789 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e39a227-64b4-47d2-9c55-1665931d2b0d None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:52.200226 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e39a227-64b4-47d2-9c55-1665931d2b0d None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:52.200346 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e39a227-64b4-47d2-9c55-1665931d2b0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:52.218739 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-1e39a227-64b4-47d2-9c55-1665931d2b0d None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:32:52.219923 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-1e39a227-64b4-47d2-9c55-1665931d2b0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:32:52.260781 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:52.293686 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 580/1136] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:32:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:32:52.319018 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b889d918-c30c-4e43-87cb-23f940f3d8ed None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:52.418215 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b889d918-c30c-4e43-87cb-23f940f3d8ed None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:52.451984 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd13d59-8110-4522-a7c6-1f738a72d566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:52.452543 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd13d59-8110-4522-a7c6-1f738a72d566 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:52.452823 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd13d59-8110-4522-a7c6-1f738a72d566 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:52.453094 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd13d59-8110-4522-a7c6-1f738a72d566 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:52.454287 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd13d59-8110-4522-a7c6-1f738a72d566 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:52.455713 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd13d59-8110-4522-a7c6-1f738a72d566 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:52.457321 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 557/1137] 10.222.0.89 () {56 vars in 1108 bytes} [Thu Jul 28 17:32:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:32:52.463069 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e39a227-64b4-47d2-9c55-1665931d2b0d None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:52.574995 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e39a227-64b4-47d2-9c55-1665931d2b0d None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:52.607894 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bb7945-d3bf-4987-886f-0f1afd899f1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:52.608498 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bb7945-d3bf-4987-886f-0f1afd899f1c None admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:52.608778 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bb7945-d3bf-4987-886f-0f1afd899f1c None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:52.609633 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bb7945-d3bf-4987-886f-0f1afd899f1c None admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:52.611097 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bb7945-d3bf-4987-886f-0f1afd899f1c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:52.612588 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bb7945-d3bf-4987-886f-0f1afd899f1c None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:52.634718 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 581/1138] 10.222.0.89 () {60 vars in 1132 bytes} [Thu Jul 28 17:32:52 2022] POST /identity/v3/users => generated 286 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 17:32:54.694546 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8322b65-c824-46d3-8866-a98c8cccad8a None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:54.695080 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8322b65-c824-46d3-8866-a98c8cccad8a None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:54.695080 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8322b65-c824-46d3-8866-a98c8cccad8a None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:54.696460 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 558/1139] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:32:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:32:54.702101 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f293a10c-0550-4061-ae18-bec22b2e67a0 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:54.702534 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f293a10c-0550-4061-ae18-bec22b2e67a0 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:54.702653 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f293a10c-0550-4061-ae18-bec22b2e67a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:54.720881 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-f293a10c-0550-4061-ae18-bec22b2e67a0 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:32:54.722100 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-f293a10c-0550-4061-ae18-bec22b2e67a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:32:54.749783 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:54.790940 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 582/1140] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:32:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:32:54.816201 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8322b65-c824-46d3-8866-a98c8cccad8a None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:54.831021 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:54.928282 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8322b65-c824-46d3-8866-a98c8cccad8a None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:54.961755 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89672cc-6bec-49b0-be83-ac7c02525a44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:54.962319 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89672cc-6bec-49b0-be83-ac7c02525a44 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:54.962597 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89672cc-6bec-49b0-be83-ac7c02525a44 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:54.962864 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89672cc-6bec-49b0-be83-ac7c02525a44 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:54.973375 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89672cc-6bec-49b0-be83-ac7c02525a44 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:32:54.978233 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-e89672cc-6bec-49b0-be83-ac7c02525a44 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:32:54.978233 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:32:54.978233 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:32:54.978233 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:32:54.978233 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:32:54.978233 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:32:54.978233 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:32:54.978233 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:32:54.978233 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:32:54.978233 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:32:54.978233 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:32:54.978233 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:32:54.978233 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:32:54.978233 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:32:54.978233 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:32:54.978233 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:32:54.979543 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:32:54.979543 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:32:54.979543 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:32:54.979543 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:32:54.979543 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:32:54.979543 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:32:54.979543 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:32:54.979543 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:32:54.979543 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:32:54.979543 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:32:54.979543 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:32:54.979543 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:32:54.979543 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:32:54.979543 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:32:54.979543 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:32:54.979543 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:32:54.980755 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:32:54.980755 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:32:54.980755 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:32:54.980755 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:32:54.980755 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:32:54.980755 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:32:54.980755 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:32:54.980755 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:32:54.980755 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:32:54.980755 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:32:54.980755 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:32:54.980755 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:32:54.980755 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 559/1141] 10.222.0.89 () {56 vars in 1096 bytes} [Thu Jul 28 17:32:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:32:54.986309 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f293a10c-0550-4061-ae18-bec22b2e67a0 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:55.089434 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f293a10c-0550-4061-ae18-bec22b2e67a0 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:55.123432 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e11449-bba2-40a5-8667-02e8d631f81f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:55.124047 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e11449-bba2-40a5-8667-02e8d631f81f None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:55.124322 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e11449-bba2-40a5-8667-02e8d631f81f None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:55.124598 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e11449-bba2-40a5-8667-02e8d631f81f None admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:55.125338 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e11449-bba2-40a5-8667-02e8d631f81f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:32:55.125646 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e11449-bba2-40a5-8667-02e8d631f81f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:55.127012 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e11449-bba2-40a5-8667-02e8d631f81f None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:55.135083 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 583/1142] 10.222.0.89 () {56 vars in 1099 bytes} [Thu Jul 28 17:32:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:32:55.140990 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89672cc-6bec-49b0-be83-ac7c02525a44 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:55.210107 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:55.240525 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89672cc-6bec-49b0-be83-ac7c02525a44 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:55.277811 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5332a8-cf0d-478e-90e6-1ed2e9469ba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:55.278447 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5332a8-cf0d-478e-90e6-1ed2e9469ba4 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:55.278726 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5332a8-cf0d-478e-90e6-1ed2e9469ba4 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:55.278997 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5332a8-cf0d-478e-90e6-1ed2e9469ba4 None admin] PATH_INFO: `/v3/users/bf4cdfa925424dff8195f4fdc0a9966e` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:55.292929 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5332a8-cf0d-478e-90e6-1ed2e9469ba4 None admin] RBAC: Authorizing `identity:get_user(user_id=bf4cdfa925424dff8195f4fdc0a9966e)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:55.294407 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5332a8-cf0d-478e-90e6-1ed2e9469ba4 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:55.296398 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 560/1143] 10.222.0.89 () {56 vars in 1177 bytes} [Thu Jul 28 17:32:55 2022] GET /identity/v3/users/bf4cdfa925424dff8195f4fdc0a9966e => generated 286 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:32:55.302590 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e11449-bba2-40a5-8667-02e8d631f81f None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:55.332080 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:55.410971 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e11449-bba2-40a5-8667-02e8d631f81f None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:55.445454 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aec3bae-5eab-44c6-b2b1-5922ab2c9a71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:55.446082 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aec3bae-5eab-44c6-b2b1-5922ab2c9a71 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:55.446368 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aec3bae-5eab-44c6-b2b1-5922ab2c9a71 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:55.446645 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aec3bae-5eab-44c6-b2b1-5922ab2c9a71 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:55.527196 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aec3bae-5eab-44c6-b2b1-5922ab2c9a71 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:55.528547 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aec3bae-5eab-44c6-b2b1-5922ab2c9a71 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:55.554177 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:55.632942 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 584/1144] 10.222.0.89 () {58 vars in 1282 bytes} [Thu Jul 28 17:32:55 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 17:32:55.639985 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5332a8-cf0d-478e-90e6-1ed2e9469ba4 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:55.747261 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5332a8-cf0d-478e-90e6-1ed2e9469ba4 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:55.781906 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:55.788329 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047fadee-fc4c-47e3-88aa-8638cbd38edb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:55.788989 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047fadee-fc4c-47e3-88aa-8638cbd38edb None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:55.789282 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047fadee-fc4c-47e3-88aa-8638cbd38edb None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:55.789555 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047fadee-fc4c-47e3-88aa-8638cbd38edb None admin] PATH_INFO: `/v3/projects/edaf20daa8d844f8b3203160c2e98ad4` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:55.790738 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047fadee-fc4c-47e3-88aa-8638cbd38edb None admin] RBAC: Authorizing `identity:get_project(project_id=edaf20daa8d844f8b3203160c2e98ad4)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:55.792165 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047fadee-fc4c-47e3-88aa-8638cbd38edb None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:55.793884 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 561/1145] 10.222.0.89 () {56 vars in 1186 bytes} [Thu Jul 28 17:32:55 2022] GET /identity/v3/projects/edaf20daa8d844f8b3203160c2e98ad4 => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:32:55.800230 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aec3bae-5eab-44c6-b2b1-5922ab2c9a71 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:55.913548 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aec3bae-5eab-44c6-b2b1-5922ab2c9a71 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:55.938127 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:55.949116 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa84e3c-261c-4fcf-9970-6d2364844b51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:55.949755 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa84e3c-261c-4fcf-9970-6d2364844b51 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:55.950035 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa84e3c-261c-4fcf-9970-6d2364844b51 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:55.950309 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa84e3c-261c-4fcf-9970-6d2364844b51 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:55.951756 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa84e3c-261c-4fcf-9970-6d2364844b51 None admin] RBAC: Adding query filter params (role.id=73e457231a824ea98c28d22610be640c, scope.project.id=edaf20daa8d844f8b3203160c2e98ad4, user.id=bf4cdfa925424dff8195f4fdc0a9966e) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:32:55.952255 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa84e3c-261c-4fcf-9970-6d2364844b51 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:55.954150 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa84e3c-261c-4fcf-9970-6d2364844b51 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:55.959582 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 585/1146] 10.222.0.89 () {56 vars in 1374 bytes} [Thu Jul 28 17:32:55 2022] GET /identity/v3/role_assignments?user.id=bf4cdfa925424dff8195f4fdc0a9966e&scope.project.id=edaf20daa8d844f8b3203160c2e98ad4&role.id=73e457231a824ea98c28d22610be640c => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:32:57.938209 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed7f74e-ac16-4364-854d-4911d3405abe None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:57.938802 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed7f74e-ac16-4364-854d-4911d3405abe None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:57.938802 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed7f74e-ac16-4364-854d-4911d3405abe None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:57.940220 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 562/1147] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:32:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:32:57.945756 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e5053a-dcc3-4138-82b3-71aba4ce3372 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:57.946194 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e5053a-dcc3-4138-82b3-71aba4ce3372 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:57.946296 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e5053a-dcc3-4138-82b3-71aba4ce3372 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:57.964929 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-b1e5053a-dcc3-4138-82b3-71aba4ce3372 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:32:57.966170 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-b1e5053a-dcc3-4138-82b3-71aba4ce3372 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:32:58.031453 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 586/1148] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:32:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:32:58.053924 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed7f74e-ac16-4364-854d-4911d3405abe None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:58.153664 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed7f74e-ac16-4364-854d-4911d3405abe None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:58.187208 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ea77e9-6122-46d3-9974-6775af71480e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:58.187851 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ea77e9-6122-46d3-9974-6775af71480e None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:58.188130 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ea77e9-6122-46d3-9974-6775af71480e None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:58.188403 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ea77e9-6122-46d3-9974-6775af71480e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:58.198396 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ea77e9-6122-46d3-9974-6775af71480e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:32:58.204072 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-f5ea77e9-6122-46d3-9974-6775af71480e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:32:58.204072 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:32:58.204072 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:32:58.204072 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:32:58.204072 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:32:58.204072 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:32:58.204072 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:32:58.204072 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:32:58.204072 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:32:58.204072 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:32:58.204072 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:32:58.204072 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:32:58.204072 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:32:58.204072 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:32:58.204072 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:32:58.204072 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:32:58.205189 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:32:58.205189 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:32:58.205189 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:32:58.205189 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:32:58.205189 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:32:58.205189 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:32:58.205189 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:32:58.205189 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:32:58.205189 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:32:58.205189 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:32:58.205189 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:32:58.205189 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:32:58.205189 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:32:58.205189 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:32:58.205189 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:32:58.205189 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:32:58.206244 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:32:58.206244 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:32:58.206244 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:32:58.206244 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:32:58.206244 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:32:58.206244 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:32:58.206244 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:32:58.206244 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:32:58.206244 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:32:58.206244 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:32:58.206244 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:32:58.206244 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:32:58.206244 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 563/1149] 10.222.0.89 () {56 vars in 1096 bytes} [Thu Jul 28 17:32:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:32:58.211386 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e5053a-dcc3-4138-82b3-71aba4ce3372 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:58.306972 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e5053a-dcc3-4138-82b3-71aba4ce3372 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:58.329262 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:58.351631 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e6dbc9-06ce-4864-adf3-4a59d725a240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:58.352510 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e6dbc9-06ce-4864-adf3-4a59d725a240 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:58.352921 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e6dbc9-06ce-4864-adf3-4a59d725a240 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:58.353369 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e6dbc9-06ce-4864-adf3-4a59d725a240 None admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:58.354816 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e6dbc9-06ce-4864-adf3-4a59d725a240 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:32:58.354944 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e6dbc9-06ce-4864-adf3-4a59d725a240 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:58.357054 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e6dbc9-06ce-4864-adf3-4a59d725a240 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:58.368052 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 587/1150] 10.222.0.89 () {56 vars in 1099 bytes} [Thu Jul 28 17:32:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:32:58.375233 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ea77e9-6122-46d3-9974-6775af71480e None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:58.432015 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:58.473650 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ea77e9-6122-46d3-9974-6775af71480e None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:58.507410 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf69d2db-ae5b-460c-8fa1-61cd746df9ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:58.508045 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf69d2db-ae5b-460c-8fa1-61cd746df9ad None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:58.508325 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf69d2db-ae5b-460c-8fa1-61cd746df9ad None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:58.508594 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf69d2db-ae5b-460c-8fa1-61cd746df9ad None admin] PATH_INFO: `/v3/users/bf4cdfa925424dff8195f4fdc0a9966e` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:58.509770 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf69d2db-ae5b-460c-8fa1-61cd746df9ad None admin] RBAC: Authorizing `identity:get_user(user_id=bf4cdfa925424dff8195f4fdc0a9966e)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:58.511191 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf69d2db-ae5b-460c-8fa1-61cd746df9ad None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:58.512786 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 564/1151] 10.222.0.89 () {56 vars in 1177 bytes} [Thu Jul 28 17:32:58 2022] GET /identity/v3/users/bf4cdfa925424dff8195f4fdc0a9966e => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:32:58.518732 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e6dbc9-06ce-4864-adf3-4a59d725a240 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:58.616038 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e6dbc9-06ce-4864-adf3-4a59d725a240 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:58.649888 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14eb9915-6ed2-4503-8e1c-4b98c1b8f8b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:58.650512 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eb9915-6ed2-4503-8e1c-4b98c1b8f8b0 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:58.650791 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eb9915-6ed2-4503-8e1c-4b98c1b8f8b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:58.651066 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eb9915-6ed2-4503-8e1c-4b98c1b8f8b0 None admin] PATH_INFO: `/v3/projects/edaf20daa8d844f8b3203160c2e98ad4` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:58.652210 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14eb9915-6ed2-4503-8e1c-4b98c1b8f8b0 None admin] RBAC: Authorizing `identity:get_project(project_id=edaf20daa8d844f8b3203160c2e98ad4)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:58.653760 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14eb9915-6ed2-4503-8e1c-4b98c1b8f8b0 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:58.656109 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 588/1152] 10.222.0.89 () {56 vars in 1186 bytes} [Thu Jul 28 17:32:58 2022] GET /identity/v3/projects/edaf20daa8d844f8b3203160c2e98ad4 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:32:58.662380 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf69d2db-ae5b-460c-8fa1-61cd746df9ad None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:32:58.761713 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf69d2db-ae5b-460c-8fa1-61cd746df9ad None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:32:58.788041 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:32:58.796945 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7a5da3-76a7-4b2d-9e8a-09e38f7a17be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:32:58.797646 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7a5da3-76a7-4b2d-9e8a-09e38f7a17be None admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:32:58.797928 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7a5da3-76a7-4b2d-9e8a-09e38f7a17be None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:32:58.798206 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7a5da3-76a7-4b2d-9e8a-09e38f7a17be None admin] PATH_INFO: `/v3/projects/edaf20daa8d844f8b3203160c2e98ad4/users/bf4cdfa925424dff8195f4fdc0a9966e/roles/73e457231a824ea98c28d22610be640c` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:32:58.800183 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7a5da3-76a7-4b2d-9e8a-09e38f7a17be None admin] RBAC: Authorizing `identity:create_grant(project_id=edaf20daa8d844f8b3203160c2e98ad4, user_id=bf4cdfa925424dff8195f4fdc0a9966e, role_id=73e457231a824ea98c28d22610be640c)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:32:58.801831 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7a5da3-76a7-4b2d-9e8a-09e38f7a17be None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:32:58.815163 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 565/1153] 10.222.0.89 () {58 vars in 1439 bytes} [Thu Jul 28 17:32:58 2022] PUT /identity/v3/projects/edaf20daa8d844f8b3203160c2e98ad4/users/bf4cdfa925424dff8195f4fdc0a9966e/roles/73e457231a824ea98c28d22610be640c => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:33:00.819254 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81409b7-c482-497f-a851-35b9a0ea1b4f None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:33:00.819819 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81409b7-c482-497f-a851-35b9a0ea1b4f None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:33:00.819819 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81409b7-c482-497f-a851-35b9a0ea1b4f None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:33:00.821148 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 589/1154] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:33:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:33:00.826930 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8754ddb1-4ba6-4f74-b203-b93c3c6d47c8 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:33:00.827410 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8754ddb1-4ba6-4f74-b203-b93c3c6d47c8 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:33:00.827691 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8754ddb1-4ba6-4f74-b203-b93c3c6d47c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:33:00.846218 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-8754ddb1-4ba6-4f74-b203-b93c3c6d47c8 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:33:00.847401 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-8754ddb1-4ba6-4f74-b203-b93c3c6d47c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:33:00.912515 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 566/1155] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:33:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:33:00.936397 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81409b7-c482-497f-a851-35b9a0ea1b4f None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:33:00.986105 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:33:01.038773 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81409b7-c482-497f-a851-35b9a0ea1b4f None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:33:01.072227 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c498b9-d6ee-44f3-949c-3ca6d2da0b5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:33:01.072776 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c498b9-d6ee-44f3-949c-3ca6d2da0b5c None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:33:01.073058 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c498b9-d6ee-44f3-949c-3ca6d2da0b5c None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:33:01.073362 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c498b9-d6ee-44f3-949c-3ca6d2da0b5c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:33:01.084576 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c498b9-d6ee-44f3-949c-3ca6d2da0b5c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:33:01.089471 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-e7c498b9-d6ee-44f3-949c-3ca6d2da0b5c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:33:01.089471 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:33:01.089471 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:33:01.089471 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:33:01.089471 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:33:01.089471 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:33:01.089471 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:33:01.089471 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:33:01.089471 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:33:01.089471 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:33:01.089471 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:33:01.089471 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:33:01.089471 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:33:01.089471 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:33:01.089471 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:33:01.089471 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:33:01.090583 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:33:01.090583 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:33:01.090583 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:33:01.090583 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:33:01.090583 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:33:01.090583 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:33:01.090583 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:33:01.090583 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:33:01.090583 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:33:01.090583 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:33:01.090583 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:33:01.090583 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( Jul 28 17:33:01.090583 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:33:01.090583 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:33:01.090583 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:33:01.090583 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:33:01.091460 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:33:01.091460 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:33:01.091460 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:33:01.091460 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:33:01.091460 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:33:01.091460 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:33:01.091460 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:33:01.091460 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:33:01.091460 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:33:01.091460 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:33:01.091460 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:33:01.091460 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:33:01.091460 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 590/1156] 10.222.0.89 () {56 vars in 1096 bytes} [Thu Jul 28 17:33:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:33:01.096754 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8754ddb1-4ba6-4f74-b203-b93c3c6d47c8 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:33:01.193026 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8754ddb1-4ba6-4f74-b203-b93c3c6d47c8 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:33:01.209731 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:33:01.227659 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61164d53-ea18-4e68-a8be-2e389791d620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:33:01.228268 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61164d53-ea18-4e68-a8be-2e389791d620 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:33:01.228548 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61164d53-ea18-4e68-a8be-2e389791d620 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:33:01.228823 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61164d53-ea18-4e68-a8be-2e389791d620 None admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:33:01.229558 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61164d53-ea18-4e68-a8be-2e389791d620 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:33:01.229861 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61164d53-ea18-4e68-a8be-2e389791d620 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:33:01.231221 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61164d53-ea18-4e68-a8be-2e389791d620 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:33:01.239170 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 567/1157] 10.222.0.89 () {56 vars in 1099 bytes} [Thu Jul 28 17:33:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:33:01.244953 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c498b9-d6ee-44f3-949c-3ca6d2da0b5c None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:33:01.343697 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c498b9-d6ee-44f3-949c-3ca6d2da0b5c None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:33:01.352282 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:33:01.379096 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e464f9-b5c4-48b6-85b2-520114dee9e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:33:01.379746 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e464f9-b5c4-48b6-85b2-520114dee9e2 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:33:01.380029 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e464f9-b5c4-48b6-85b2-520114dee9e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:33:01.380305 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e464f9-b5c4-48b6-85b2-520114dee9e2 None admin] PATH_INFO: `/v3/users/bf4cdfa925424dff8195f4fdc0a9966e` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:33:01.381496 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e464f9-b5c4-48b6-85b2-520114dee9e2 None admin] RBAC: Authorizing `identity:get_user(user_id=bf4cdfa925424dff8195f4fdc0a9966e)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:33:01.382932 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e464f9-b5c4-48b6-85b2-520114dee9e2 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:33:01.384523 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 591/1158] 10.222.0.89 () {56 vars in 1177 bytes} [Thu Jul 28 17:33:01 2022] GET /identity/v3/users/bf4cdfa925424dff8195f4fdc0a9966e => generated 286 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:33:01.390439 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61164d53-ea18-4e68-a8be-2e389791d620 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:33:01.486467 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61164d53-ea18-4e68-a8be-2e389791d620 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:33:01.518952 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca050c00-8f45-45e8-bb86-bfa77f16a055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:33:01.519556 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca050c00-8f45-45e8-bb86-bfa77f16a055 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:33:01.519838 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca050c00-8f45-45e8-bb86-bfa77f16a055 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:33:01.520111 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca050c00-8f45-45e8-bb86-bfa77f16a055 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:33:01.564551 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:33:01.586197 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca050c00-8f45-45e8-bb86-bfa77f16a055 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:33:01.587637 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca050c00-8f45-45e8-bb86-bfa77f16a055 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:33:01.683112 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 568/1159] 10.222.0.89 () {58 vars in 1282 bytes} [Thu Jul 28 17:33:01 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 17:33:01.689898 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e464f9-b5c4-48b6-85b2-520114dee9e2 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:33:01.786770 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e464f9-b5c4-48b6-85b2-520114dee9e2 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:33:01.819251 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dbd3d1-8a81-4695-829c-00ed8e760772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:33:01.819873 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dbd3d1-8a81-4695-829c-00ed8e760772 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:33:01.820418 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dbd3d1-8a81-4695-829c-00ed8e760772 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:33:01.820418 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dbd3d1-8a81-4695-829c-00ed8e760772 None admin] PATH_INFO: `/v3/projects/edaf20daa8d844f8b3203160c2e98ad4` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:33:01.821577 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dbd3d1-8a81-4695-829c-00ed8e760772 None admin] RBAC: Authorizing `identity:get_project(project_id=edaf20daa8d844f8b3203160c2e98ad4)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:33:01.823000 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dbd3d1-8a81-4695-829c-00ed8e760772 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:33:01.824906 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 592/1160] 10.222.0.89 () {56 vars in 1186 bytes} [Thu Jul 28 17:33:01 2022] GET /identity/v3/projects/edaf20daa8d844f8b3203160c2e98ad4 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:33:01.832051 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca050c00-8f45-45e8-bb86-bfa77f16a055 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:33:01.907547 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:33:01.928626 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca050c00-8f45-45e8-bb86-bfa77f16a055 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:33:01.962150 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7409cf65-adb8-4cc9-aa3a-2dacab119e49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:33:01.962692 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7409cf65-adb8-4cc9-aa3a-2dacab119e49 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:33:01.962970 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7409cf65-adb8-4cc9-aa3a-2dacab119e49 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:33:01.963241 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7409cf65-adb8-4cc9-aa3a-2dacab119e49 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:33:01.964027 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7409cf65-adb8-4cc9-aa3a-2dacab119e49 None admin] RBAC: Adding query filter params (role.id=73e457231a824ea98c28d22610be640c, scope.project.id=edaf20daa8d844f8b3203160c2e98ad4, user.id=bf4cdfa925424dff8195f4fdc0a9966e) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:33:01.964334 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7409cf65-adb8-4cc9-aa3a-2dacab119e49 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:33:01.965796 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7409cf65-adb8-4cc9-aa3a-2dacab119e49 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:33:01.971480 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 569/1161] 10.222.0.89 () {56 vars in 1374 bytes} [Thu Jul 28 17:33:01 2022] GET /identity/v3/role_assignments?user.id=bf4cdfa925424dff8195f4fdc0a9966e&scope.project.id=edaf20daa8d844f8b3203160c2e98ad4&role.id=73e457231a824ea98c28d22610be640c => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:33:03.978779 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ddd86e-2fce-4508-847b-bb20201f00a7 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:33:03.979351 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ddd86e-2fce-4508-847b-bb20201f00a7 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:33:03.979351 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ddd86e-2fce-4508-847b-bb20201f00a7 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:33:03.980687 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 593/1162] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:33:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:33:03.986873 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd3ab92-8bbf-464f-a9d6-2f06f586ef0c None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:33:03.987325 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd3ab92-8bbf-464f-a9d6-2f06f586ef0c None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:33:03.987429 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd3ab92-8bbf-464f-a9d6-2f06f586ef0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:33:04.006052 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-cbd3ab92-8bbf-464f-a9d6-2f06f586ef0c None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:33:04.007241 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-cbd3ab92-8bbf-464f-a9d6-2f06f586ef0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:33:04.073372 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 570/1163] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:33:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:33:04.095622 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ddd86e-2fce-4508-847b-bb20201f00a7 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:33:04.141508 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:33:04.196471 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ddd86e-2fce-4508-847b-bb20201f00a7 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:33:04.230390 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9c7deb-97a4-4e7c-9b0f-d96beb319712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:33:04.230991 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9c7deb-97a4-4e7c-9b0f-d96beb319712 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:33:04.231271 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9c7deb-97a4-4e7c-9b0f-d96beb319712 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:33:04.231542 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9c7deb-97a4-4e7c-9b0f-d96beb319712 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:33:04.297652 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9c7deb-97a4-4e7c-9b0f-d96beb319712 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:33:04.299087 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9c7deb-97a4-4e7c-9b0f-d96beb319712 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:33:04.340942 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:33:04.394824 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 594/1164] 10.222.0.89 () {58 vars in 1282 bytes} [Thu Jul 28 17:33:04 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 17:33:04.400767 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd3ab92-8bbf-464f-a9d6-2f06f586ef0c None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:33:04.469237 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:33:04.497808 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd3ab92-8bbf-464f-a9d6-2f06f586ef0c None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:33:04.532176 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c5715e-8d88-4a63-85f7-06a318a4b715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:33:04.532795 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c5715e-8d88-4a63-85f7-06a318a4b715 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:33:04.533074 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c5715e-8d88-4a63-85f7-06a318a4b715 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:33:04.533376 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c5715e-8d88-4a63-85f7-06a318a4b715 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:33:04.539926 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c5715e-8d88-4a63-85f7-06a318a4b715 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:33:04.541397 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c5715e-8d88-4a63-85f7-06a318a4b715 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:33:04.553286 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-f9c5715e-8d88-4a63-85f7-06a318a4b715 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 28 17:33:04.553286 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:33:04.553286 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 28 17:33:04.553286 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 28 17:33:04.553286 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 17:33:04.553286 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 17:33:04.553286 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 17:33:04.553286 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 17:33:04.553286 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 28 17:33:04.553286 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:33:04.553286 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 17:33:04.553286 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:33:04.553286 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:33:04.553286 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:33:04.553286 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:33:04.553286 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:33:04.553286 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:33:04.554572 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:33:04.554572 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:33:04.554572 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:33:04.554572 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:33:04.554572 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:33:04.554572 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:33:04.554572 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:33:04.554572 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:33:04.554572 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:33:04.554572 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:33:04.554572 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:33:04.554572 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:33:04.554572 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:33:04.554572 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:33:04.554572 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:33:04.554572 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:33:04.555744 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:33:04.555744 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:33:04.555744 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:33:04.555744 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:33:04.555744 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:33:04.555744 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:33:04.555744 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:33:04.555744 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:33:04.555744 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:33:04.555744 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:33:04.555744 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 28 17:33:04.555744 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:33:04.555744 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 28 17:33:04.555744 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:33:04.555744 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 571/1165] 10.222.0.89 () {56 vars in 1117 bytes} [Thu Jul 28 17:33:04 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:33:04.561247 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9c7deb-97a4-4e7c-9b0f-d96beb319712 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:33:04.658461 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9c7deb-97a4-4e7c-9b0f-d96beb319712 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:33:04.691625 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f911fa8-50dd-465c-b008-4c238baf2910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:33:04.692229 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f911fa8-50dd-465c-b008-4c238baf2910 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:33:04.692510 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f911fa8-50dd-465c-b008-4c238baf2910 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:33:04.692785 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f911fa8-50dd-465c-b008-4c238baf2910 None admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:33:04.693489 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f911fa8-50dd-465c-b008-4c238baf2910 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:33:04.693795 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f911fa8-50dd-465c-b008-4c238baf2910 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:33:04.695171 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f911fa8-50dd-465c-b008-4c238baf2910 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:33:04.702099 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 595/1166] 10.222.0.89 () {56 vars in 1115 bytes} [Thu Jul 28 17:33:04 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:33:06.686023 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3777c4d-4a2e-456d-a8d9-611e024034a7 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:33:06.686556 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3777c4d-4a2e-456d-a8d9-611e024034a7 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:33:06.686556 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3777c4d-4a2e-456d-a8d9-611e024034a7 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:33:06.687912 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 572/1167] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:33:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:33:06.693020 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a54576c-5d50-40e1-91cb-5dcc5724e98a None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:33:06.693432 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a54576c-5d50-40e1-91cb-5dcc5724e98a None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:33:06.693600 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a54576c-5d50-40e1-91cb-5dcc5724e98a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:33:06.712223 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-0a54576c-5d50-40e1-91cb-5dcc5724e98a None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:33:06.713499 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-0a54576c-5d50-40e1-91cb-5dcc5724e98a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:33:06.715438 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:33:06.780561 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 596/1168] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:33:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:33:06.803127 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3777c4d-4a2e-456d-a8d9-611e024034a7 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:33:06.907089 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3777c4d-4a2e-456d-a8d9-611e024034a7 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:33:06.934322 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:33:06.940090 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4742e70-3b70-401c-83e3-9bb2608e4bcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:33:06.940640 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4742e70-3b70-401c-83e3-9bb2608e4bcd None admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:33:06.940921 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4742e70-3b70-401c-83e3-9bb2608e4bcd None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:33:06.941223 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4742e70-3b70-401c-83e3-9bb2608e4bcd None admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:33:06.941948 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4742e70-3b70-401c-83e3-9bb2608e4bcd None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:33:06.943295 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4742e70-3b70-401c-83e3-9bb2608e4bcd None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:33:06.959089 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 573/1169] 10.222.0.89 () {60 vars in 1138 bytes} [Thu Jul 28 17:33:06 2022] POST /identity/v3/domains => generated 267 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:33:09.001014 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c214017-7ce3-4535-8be3-545a11ee4afd None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:33:09.001692 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c214017-7ce3-4535-8be3-545a11ee4afd None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:33:09.001858 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c214017-7ce3-4535-8be3-545a11ee4afd None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:33:09.003052 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 597/1170] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:33:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:33:09.008119 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fe252d-b33f-4a36-a5f2-6c0290f4404e None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:33:09.008514 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fe252d-b33f-4a36-a5f2-6c0290f4404e None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:33:09.008666 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fe252d-b33f-4a36-a5f2-6c0290f4404e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:33:09.027233 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-72fe252d-b33f-4a36-a5f2-6c0290f4404e None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:33:09.028407 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-72fe252d-b33f-4a36-a5f2-6c0290f4404e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:33:09.095740 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 574/1171] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:33:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:33:09.119012 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c214017-7ce3-4535-8be3-545a11ee4afd None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:33:09.219098 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c214017-7ce3-4535-8be3-545a11ee4afd None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:33:09.252395 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5269d22-71b7-4ff5-b190-d5da86289248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:33:09.253021 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5269d22-71b7-4ff5-b190-d5da86289248 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:33:09.253333 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5269d22-71b7-4ff5-b190-d5da86289248 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:33:09.253610 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5269d22-71b7-4ff5-b190-d5da86289248 None admin] PATH_INFO: `/v3/domains/2c5f609204f1447e85c8d14d3af23f02` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:33:09.259951 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:33:09.264893 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5269d22-71b7-4ff5-b190-d5da86289248 None admin] RBAC: Authorizing `identity:get_domain(domain_id=2c5f609204f1447e85c8d14d3af23f02)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:33:09.266363 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5269d22-71b7-4ff5-b190-d5da86289248 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:33:09.268008 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 598/1172] 10.222.0.89 () {56 vars in 1183 bytes} [Thu Jul 28 17:33:09 2022] GET /identity/v3/domains/2c5f609204f1447e85c8d14d3af23f02 => generated 267 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:33:09.274019 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fe252d-b33f-4a36-a5f2-6c0290f4404e None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:33:09.369293 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fe252d-b33f-4a36-a5f2-6c0290f4404e None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:33:09.391891 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:33:09.403627 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42fc713-3775-4036-834e-b530cf07b162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:33:09.404727 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42fc713-3775-4036-834e-b530cf07b162 None admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:33:09.405039 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42fc713-3775-4036-834e-b530cf07b162 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:33:09.405912 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42fc713-3775-4036-834e-b530cf07b162 None admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:33:09.406149 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42fc713-3775-4036-834e-b530cf07b162 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:33:09.407564 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42fc713-3775-4036-834e-b530cf07b162 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:33:09.433755 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 575/1173] 10.222.0.89 () {60 vars in 1141 bytes} [Thu Jul 28 17:33:09 2022] POST /identity/v3/projects => generated 361 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:33:11.547326 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930c828c-aa6b-4fde-ad04-585174be4bb7 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:33:11.547919 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930c828c-aa6b-4fde-ad04-585174be4bb7 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:33:11.547919 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930c828c-aa6b-4fde-ad04-585174be4bb7 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:33:11.549236 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 599/1174] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:33:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:33:11.554330 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ca043e-5558-4e2a-9e71-c1a6fb3e1c07 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:33:11.554750 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ca043e-5558-4e2a-9e71-c1a6fb3e1c07 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:33:11.554894 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ca043e-5558-4e2a-9e71-c1a6fb3e1c07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:33:11.573411 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-19ca043e-5558-4e2a-9e71-c1a6fb3e1c07 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:33:11.574613 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-19ca043e-5558-4e2a-9e71-c1a6fb3e1c07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:33:11.641567 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 576/1175] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:33:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:33:11.665926 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930c828c-aa6b-4fde-ad04-585174be4bb7 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:33:11.765820 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930c828c-aa6b-4fde-ad04-585174be4bb7 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:33:11.799330 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6831530-d425-4db6-b4fc-6b2a7a9191db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:33:11.799892 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6831530-d425-4db6-b4fc-6b2a7a9191db None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:33:11.800174 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6831530-d425-4db6-b4fc-6b2a7a9191db None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:33:11.800449 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6831530-d425-4db6-b4fc-6b2a7a9191db None admin] PATH_INFO: `/v3/domains/2c5f609204f1447e85c8d14d3af23f02` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:33:11.801634 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6831530-d425-4db6-b4fc-6b2a7a9191db None admin] RBAC: Authorizing `identity:get_domain(domain_id=2c5f609204f1447e85c8d14d3af23f02)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:33:11.803062 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6831530-d425-4db6-b4fc-6b2a7a9191db None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:33:11.804649 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 600/1176] 10.222.0.89 () {56 vars in 1183 bytes} [Thu Jul 28 17:33:11 2022] GET /identity/v3/domains/2c5f609204f1447e85c8d14d3af23f02 => generated 267 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:33:11.811305 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ca043e-5558-4e2a-9e71-c1a6fb3e1c07 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:33:11.908282 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ca043e-5558-4e2a-9e71-c1a6fb3e1c07 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:33:11.910565 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:33:11.943126 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbc58a9-a29b-42fe-b101-59fddff8a8c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:33:11.943744 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbc58a9-a29b-42fe-b101-59fddff8a8c0 None admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:33:11.944021 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbc58a9-a29b-42fe-b101-59fddff8a8c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:33:11.944294 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbc58a9-a29b-42fe-b101-59fddff8a8c0 None admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:33:11.944982 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbc58a9-a29b-42fe-b101-59fddff8a8c0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:33:11.946423 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbc58a9-a29b-42fe-b101-59fddff8a8c0 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:33:11.967993 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 577/1177] 10.222.0.89 () {60 vars in 1132 bytes} [Thu Jul 28 17:33:11 2022] POST /identity/v3/users => generated 311 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 17:33:14.020322 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec02b52-d48c-48b5-b4cd-a2e06b7fe0db None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:33:14.020875 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec02b52-d48c-48b5-b4cd-a2e06b7fe0db None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:33:14.020875 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec02b52-d48c-48b5-b4cd-a2e06b7fe0db None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:33:14.022227 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 601/1178] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:33:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:33:14.027274 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42433ed-cdce-46fc-b06f-77a606ce0edf None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:33:14.027707 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42433ed-cdce-46fc-b06f-77a606ce0edf None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:33:14.027807 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42433ed-cdce-46fc-b06f-77a606ce0edf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:33:14.046163 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-d42433ed-cdce-46fc-b06f-77a606ce0edf None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:33:14.047344 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-d42433ed-cdce-46fc-b06f-77a606ce0edf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:33:14.111997 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 578/1179] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:33:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:33:14.135456 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec02b52-d48c-48b5-b4cd-a2e06b7fe0db None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:33:14.203424 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:33:14.240995 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec02b52-d48c-48b5-b4cd-a2e06b7fe0db None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:33:14.274401 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21b223a-6f85-46f2-a34d-3adcaff6cf28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:33:14.275026 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21b223a-6f85-46f2-a34d-3adcaff6cf28 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:33:14.275304 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21b223a-6f85-46f2-a34d-3adcaff6cf28 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:33:14.275574 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21b223a-6f85-46f2-a34d-3adcaff6cf28 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:33:14.285439 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21b223a-6f85-46f2-a34d-3adcaff6cf28 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:33:14.290270 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-b21b223a-6f85-46f2-a34d-3adcaff6cf28 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:33:14.290270 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:33:14.290270 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:33:14.290270 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:33:14.290270 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:33:14.290270 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:33:14.290270 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:33:14.290270 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:33:14.290270 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:33:14.290270 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:33:14.290270 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:33:14.290270 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:33:14.290270 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:33:14.290270 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:33:14.290270 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:33:14.290270 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:33:14.291438 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:33:14.291438 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:33:14.291438 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:33:14.291438 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:33:14.291438 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:33:14.291438 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:33:14.291438 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:33:14.291438 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:33:14.291438 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:33:14.291438 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:33:14.291438 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:33:14.291438 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( Jul 28 17:33:14.291438 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:33:14.291438 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:33:14.291438 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:33:14.291438 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:33:14.292717 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:33:14.292717 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:33:14.292717 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:33:14.292717 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:33:14.292717 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:33:14.292717 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:33:14.292717 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:33:14.292717 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:33:14.292717 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:33:14.292717 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:33:14.292717 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:33:14.292717 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:33:14.292717 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 602/1180] 10.222.0.89 () {56 vars in 1096 bytes} [Thu Jul 28 17:33:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:33:14.298115 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42433ed-cdce-46fc-b06f-77a606ce0edf None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:33:14.384084 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:33:14.394396 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42433ed-cdce-46fc-b06f-77a606ce0edf None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:33:14.427860 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb5537e-e43d-42bd-ab88-9ac1e291e9df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:33:14.428465 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb5537e-e43d-42bd-ab88-9ac1e291e9df None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:33:14.428743 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb5537e-e43d-42bd-ab88-9ac1e291e9df None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:33:14.429015 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb5537e-e43d-42bd-ab88-9ac1e291e9df None admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:33:14.429755 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb5537e-e43d-42bd-ab88-9ac1e291e9df None admin] RBAC: Adding query filter params (name=admin) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:33:14.430061 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb5537e-e43d-42bd-ab88-9ac1e291e9df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:33:14.431421 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb5537e-e43d-42bd-ab88-9ac1e291e9df None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:33:14.439403 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 579/1181] 10.222.0.89 () {56 vars in 1099 bytes} [Thu Jul 28 17:33:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:33:14.445114 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21b223a-6f85-46f2-a34d-3adcaff6cf28 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:33:14.546976 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21b223a-6f85-46f2-a34d-3adcaff6cf28 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:33:14.571231 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:33:14.580642 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4feec37a-f26b-4432-b86e-8d48c399809e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:33:14.581322 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feec37a-f26b-4432-b86e-8d48c399809e None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:33:14.581623 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feec37a-f26b-4432-b86e-8d48c399809e None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:33:14.581950 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feec37a-f26b-4432-b86e-8d48c399809e None admin] PATH_INFO: `/v3/users/1aefea8c2dcb4b22bf732f2ebb0c8fe2` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:33:14.595415 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4feec37a-f26b-4432-b86e-8d48c399809e None admin] RBAC: Authorizing `identity:get_user(user_id=1aefea8c2dcb4b22bf732f2ebb0c8fe2)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:33:14.596863 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4feec37a-f26b-4432-b86e-8d48c399809e None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:33:14.598523 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 603/1182] 10.222.0.89 () {56 vars in 1177 bytes} [Thu Jul 28 17:33:14 2022] GET /identity/v3/users/1aefea8c2dcb4b22bf732f2ebb0c8fe2 => generated 311 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:33:14.604490 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb5537e-e43d-42bd-ab88-9ac1e291e9df None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:33:14.700221 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb5537e-e43d-42bd-ab88-9ac1e291e9df None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:33:14.732999 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abfe55b-5cc5-48c9-8a4b-4655dfba335d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:33:14.733698 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abfe55b-5cc5-48c9-8a4b-4655dfba335d None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:33:14.733980 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abfe55b-5cc5-48c9-8a4b-4655dfba335d None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:33:14.734248 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abfe55b-5cc5-48c9-8a4b-4655dfba335d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:33:14.754672 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:33:14.799998 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abfe55b-5cc5-48c9-8a4b-4655dfba335d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:33:14.801472 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abfe55b-5cc5-48c9-8a4b-4655dfba335d None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:33:14.899096 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 580/1183] 10.222.0.89 () {58 vars in 1282 bytes} [Thu Jul 28 17:33:14 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 17:33:14.905729 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4feec37a-f26b-4432-b86e-8d48c399809e None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:33:15.006984 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4feec37a-f26b-4432-b86e-8d48c399809e None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:33:15.039860 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ed4ebf-87df-44ca-854e-7a3bda999e0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:33:15.040501 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ed4ebf-87df-44ca-854e-7a3bda999e0d None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:33:15.040785 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ed4ebf-87df-44ca-854e-7a3bda999e0d None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:33:15.041060 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ed4ebf-87df-44ca-854e-7a3bda999e0d None admin] PATH_INFO: `/v3/projects/12c5f40927a5460ca4193afa70b5edc6` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:33:15.042259 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ed4ebf-87df-44ca-854e-7a3bda999e0d None admin] RBAC: Authorizing `identity:get_project(project_id=12c5f40927a5460ca4193afa70b5edc6)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:33:15.043700 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ed4ebf-87df-44ca-854e-7a3bda999e0d None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:33:15.046102 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 604/1184] 10.222.0.89 () {56 vars in 1186 bytes} [Thu Jul 28 17:33:14 2022] GET /identity/v3/projects/12c5f40927a5460ca4193afa70b5edc6 => generated 361 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:33:15.052275 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abfe55b-5cc5-48c9-8a4b-4655dfba335d None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:33:15.103013 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:33:15.149572 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abfe55b-5cc5-48c9-8a4b-4655dfba335d None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:33:15.183254 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6956e11-54d2-452e-b39a-6dbd10e36e69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:33:15.183862 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6956e11-54d2-452e-b39a-6dbd10e36e69 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:33:15.184138 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6956e11-54d2-452e-b39a-6dbd10e36e69 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:33:15.184408 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6956e11-54d2-452e-b39a-6dbd10e36e69 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:33:15.185213 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6956e11-54d2-452e-b39a-6dbd10e36e69 None admin] RBAC: Adding query filter params (role.id=73e457231a824ea98c28d22610be640c, scope.project.id=12c5f40927a5460ca4193afa70b5edc6, user.id=1aefea8c2dcb4b22bf732f2ebb0c8fe2) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:33:15.185555 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6956e11-54d2-452e-b39a-6dbd10e36e69 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:33:15.186973 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6956e11-54d2-452e-b39a-6dbd10e36e69 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:33:15.192191 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 581/1185] 10.222.0.89 () {56 vars in 1374 bytes} [Thu Jul 28 17:33:15 2022] GET /identity/v3/role_assignments?user.id=1aefea8c2dcb4b22bf732f2ebb0c8fe2&scope.project.id=12c5f40927a5460ca4193afa70b5edc6&role.id=73e457231a824ea98c28d22610be640c => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:33:17.187480 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc74615-07da-4f4c-95ff-e6cb2f4cb3ce None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:33:17.188036 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc74615-07da-4f4c-95ff-e6cb2f4cb3ce None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:33:17.188036 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc74615-07da-4f4c-95ff-e6cb2f4cb3ce None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:33:17.189400 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 605/1186] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:33:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 17:33:17.194740 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60707f92-5f35-4101-99f4-3efd686ea1ff None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:33:17.195187 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60707f92-5f35-4101-99f4-3efd686ea1ff None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:33:17.195290 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60707f92-5f35-4101-99f4-3efd686ea1ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:33:17.213308 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-60707f92-5f35-4101-99f4-3efd686ea1ff None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:33:17.214499 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-60707f92-5f35-4101-99f4-3efd686ea1ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:33:17.281034 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 582/1187] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:33:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 17:33:17.303594 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc74615-07da-4f4c-95ff-e6cb2f4cb3ce None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:33:17.356497 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:33:17.405954 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc74615-07da-4f4c-95ff-e6cb2f4cb3ce None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:33:17.439835 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecacc6e-0353-4810-b431-8d526295ce0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:33:17.440398 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecacc6e-0353-4810-b431-8d526295ce0b None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:33:17.440679 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecacc6e-0353-4810-b431-8d526295ce0b None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:33:17.440952 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecacc6e-0353-4810-b431-8d526295ce0b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:33:17.450785 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecacc6e-0353-4810-b431-8d526295ce0b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:33:17.455642 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-2ecacc6e-0353-4810-b431-8d526295ce0b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:33:17.455642 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:33:17.455642 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:33:17.455642 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:33:17.455642 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:33:17.455642 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:33:17.455642 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:33:17.455642 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:33:17.455642 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:33:17.455642 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:33:17.455642 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:33:17.455642 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:33:17.455642 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:33:17.455642 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:33:17.455642 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:33:17.455642 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:33:17.456789 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:33:17.456789 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:33:17.456789 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:33:17.456789 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:33:17.456789 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:33:17.456789 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:33:17.456789 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:33:17.456789 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:33:17.456789 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:33:17.456789 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:33:17.456789 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:33:17.456789 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( Jul 28 17:33:17.456789 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:33:17.456789 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:33:17.456789 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:33:17.456789 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:33:17.457981 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:33:17.457981 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:33:17.457981 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:33:17.457981 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:33:17.457981 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:33:17.457981 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:33:17.457981 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:33:17.457981 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:33:17.457981 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:33:17.457981 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:33:17.457981 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:33:17.457981 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:33:17.457981 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 606/1188] 10.222.0.89 () {56 vars in 1096 bytes} [Thu Jul 28 17:33:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:33:17.462826 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60707f92-5f35-4101-99f4-3efd686ea1ff None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:33:17.509134 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:33:17.562143 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60707f92-5f35-4101-99f4-3efd686ea1ff None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:33:17.594974 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f37f8ac-980c-492e-a315-f2a235697367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:33:17.595579 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f37f8ac-980c-492e-a315-f2a235697367 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:33:17.595855 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f37f8ac-980c-492e-a315-f2a235697367 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:33:17.596121 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f37f8ac-980c-492e-a315-f2a235697367 None admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:33:17.596823 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f37f8ac-980c-492e-a315-f2a235697367 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:33:17.597120 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f37f8ac-980c-492e-a315-f2a235697367 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:33:17.598500 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f37f8ac-980c-492e-a315-f2a235697367 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:33:17.606293 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 583/1189] 10.222.0.89 () {56 vars in 1099 bytes} [Thu Jul 28 17:33:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:33:17.612887 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecacc6e-0353-4810-b431-8d526295ce0b None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:33:17.712273 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecacc6e-0353-4810-b431-8d526295ce0b None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:33:17.725707 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:33:17.747038 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0cfa19-7c03-4a99-967c-ae8cc600537d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:33:17.747901 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0cfa19-7c03-4a99-967c-ae8cc600537d None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:33:17.748686 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0cfa19-7c03-4a99-967c-ae8cc600537d None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:33:17.748967 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0cfa19-7c03-4a99-967c-ae8cc600537d None admin] PATH_INFO: `/v3/users/1aefea8c2dcb4b22bf732f2ebb0c8fe2` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:33:17.750162 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0cfa19-7c03-4a99-967c-ae8cc600537d None admin] RBAC: Authorizing `identity:get_user(user_id=1aefea8c2dcb4b22bf732f2ebb0c8fe2)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:33:17.751577 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0cfa19-7c03-4a99-967c-ae8cc600537d None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:33:17.753172 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 607/1190] 10.222.0.89 () {56 vars in 1177 bytes} [Thu Jul 28 17:33:17 2022] GET /identity/v3/users/1aefea8c2dcb4b22bf732f2ebb0c8fe2 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:33:17.758930 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f37f8ac-980c-492e-a315-f2a235697367 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:33:17.863208 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f37f8ac-980c-492e-a315-f2a235697367 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:33:17.871977 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:33:17.897505 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979ed522-d8d5-4f2a-b422-9ff7484a9b39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:33:17.898140 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979ed522-d8d5-4f2a-b422-9ff7484a9b39 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:33:17.898419 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979ed522-d8d5-4f2a-b422-9ff7484a9b39 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:33:17.898693 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979ed522-d8d5-4f2a-b422-9ff7484a9b39 None admin] PATH_INFO: `/v3/projects/12c5f40927a5460ca4193afa70b5edc6` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:33:17.899842 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979ed522-d8d5-4f2a-b422-9ff7484a9b39 None admin] RBAC: Authorizing `identity:get_project(project_id=12c5f40927a5460ca4193afa70b5edc6)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:33:17.901332 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979ed522-d8d5-4f2a-b422-9ff7484a9b39 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:33:17.903004 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 584/1191] 10.222.0.89 () {56 vars in 1186 bytes} [Thu Jul 28 17:33:17 2022] GET /identity/v3/projects/12c5f40927a5460ca4193afa70b5edc6 => generated 361 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:33:17.909907 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0cfa19-7c03-4a99-967c-ae8cc600537d None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:33:18.007522 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0cfa19-7c03-4a99-967c-ae8cc600537d None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:33:18.041428 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e4cb9a-8148-40ae-9aa4-84cb06b4f483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:33:18.042084 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e4cb9a-8148-40ae-9aa4-84cb06b4f483 None admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:33:18.042367 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e4cb9a-8148-40ae-9aa4-84cb06b4f483 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:33:18.042641 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e4cb9a-8148-40ae-9aa4-84cb06b4f483 None admin] PATH_INFO: `/v3/projects/12c5f40927a5460ca4193afa70b5edc6/users/1aefea8c2dcb4b22bf732f2ebb0c8fe2/roles/73e457231a824ea98c28d22610be640c` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:33:18.044576 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e4cb9a-8148-40ae-9aa4-84cb06b4f483 None admin] RBAC: Authorizing `identity:create_grant(project_id=12c5f40927a5460ca4193afa70b5edc6, user_id=1aefea8c2dcb4b22bf732f2ebb0c8fe2, role_id=73e457231a824ea98c28d22610be640c)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:33:18.046080 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e4cb9a-8148-40ae-9aa4-84cb06b4f483 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:33:18.058402 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 608/1192] 10.222.0.89 () {58 vars in 1439 bytes} [Thu Jul 28 17:33:17 2022] PUT /identity/v3/projects/12c5f40927a5460ca4193afa70b5edc6/users/1aefea8c2dcb4b22bf732f2ebb0c8fe2/roles/73e457231a824ea98c28d22610be640c => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:33:20.049584 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6ec435-0dfc-47c6-a7e6-234f113fe189 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:33:20.050435 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6ec435-0dfc-47c6-a7e6-234f113fe189 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:33:20.050617 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6ec435-0dfc-47c6-a7e6-234f113fe189 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:33:20.051988 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 585/1193] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:33:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:33:20.057179 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac211e0-eebc-41a7-8237-afb01500cdc5 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:33:20.057591 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac211e0-eebc-41a7-8237-afb01500cdc5 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:33:20.057751 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac211e0-eebc-41a7-8237-afb01500cdc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:33:20.076773 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-eac211e0-eebc-41a7-8237-afb01500cdc5 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:33:20.078011 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-eac211e0-eebc-41a7-8237-afb01500cdc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:33:20.113015 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:33:20.146286 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 609/1194] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:33:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:33:20.170394 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6ec435-0dfc-47c6-a7e6-234f113fe189 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:33:20.270427 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6ec435-0dfc-47c6-a7e6-234f113fe189 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:33:20.302985 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2406ba-5749-42cd-afac-1dbd9c293942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:33:20.303617 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2406ba-5749-42cd-afac-1dbd9c293942 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:33:20.303900 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2406ba-5749-42cd-afac-1dbd9c293942 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:33:20.304176 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2406ba-5749-42cd-afac-1dbd9c293942 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:33:20.314978 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2406ba-5749-42cd-afac-1dbd9c293942 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:33:20.319875 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-eb2406ba-5749-42cd-afac-1dbd9c293942 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:33:20.319875 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:33:20.319875 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:33:20.319875 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:33:20.319875 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:33:20.319875 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:33:20.319875 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:33:20.319875 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:33:20.319875 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:33:20.319875 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:33:20.319875 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:33:20.319875 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:33:20.319875 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:33:20.319875 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:33:20.319875 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:33:20.319875 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:33:20.320985 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:33:20.320985 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:33:20.320985 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:33:20.320985 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:33:20.320985 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:33:20.320985 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:33:20.320985 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:33:20.320985 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:33:20.320985 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:33:20.320985 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:33:20.320985 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:33:20.320985 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:33:20.320985 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:33:20.320985 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:33:20.320985 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:33:20.320985 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:33:20.322097 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:33:20.322097 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:33:20.322097 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:33:20.322097 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:33:20.322097 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:33:20.322097 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:33:20.322097 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:33:20.322097 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:33:20.322097 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:33:20.322097 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:33:20.322097 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:33:20.322097 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:33:20.322097 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 586/1195] 10.222.0.89 () {56 vars in 1096 bytes} [Thu Jul 28 17:33:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:33:20.327621 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac211e0-eebc-41a7-8237-afb01500cdc5 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:33:20.425265 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac211e0-eebc-41a7-8237-afb01500cdc5 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:33:20.458498 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2617ca-050d-476d-88c7-a6bb3072d7b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:33:20.459106 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2617ca-050d-476d-88c7-a6bb3072d7b8 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:33:20.459387 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2617ca-050d-476d-88c7-a6bb3072d7b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:33:20.459659 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2617ca-050d-476d-88c7-a6bb3072d7b8 None admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:33:20.460368 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2617ca-050d-476d-88c7-a6bb3072d7b8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:33:20.460671 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2617ca-050d-476d-88c7-a6bb3072d7b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:33:20.462056 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2617ca-050d-476d-88c7-a6bb3072d7b8 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:33:20.470997 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 610/1196] 10.222.0.89 () {56 vars in 1099 bytes} [Thu Jul 28 17:33:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:33:20.476762 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2406ba-5749-42cd-afac-1dbd9c293942 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:33:20.511877 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:33:20.576794 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2406ba-5749-42cd-afac-1dbd9c293942 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:33:20.611677 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f7c6ce-e1e6-4917-a5a6-6c5c3ab90cd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:33:20.612304 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f7c6ce-e1e6-4917-a5a6-6c5c3ab90cd0 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:33:20.612578 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f7c6ce-e1e6-4917-a5a6-6c5c3ab90cd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:33:20.612848 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f7c6ce-e1e6-4917-a5a6-6c5c3ab90cd0 None admin] PATH_INFO: `/v3/users/1aefea8c2dcb4b22bf732f2ebb0c8fe2` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:33:20.614027 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f7c6ce-e1e6-4917-a5a6-6c5c3ab90cd0 None admin] RBAC: Authorizing `identity:get_user(user_id=1aefea8c2dcb4b22bf732f2ebb0c8fe2)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:33:20.615437 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f7c6ce-e1e6-4917-a5a6-6c5c3ab90cd0 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:33:20.617019 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 587/1197] 10.222.0.89 () {56 vars in 1177 bytes} [Thu Jul 28 17:33:20 2022] GET /identity/v3/users/1aefea8c2dcb4b22bf732f2ebb0c8fe2 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:33:20.623126 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2617ca-050d-476d-88c7-a6bb3072d7b8 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:33:20.653882 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:33:20.721874 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2617ca-050d-476d-88c7-a6bb3072d7b8 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:33:20.754791 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd47945e-2d03-4a42-afb6-daa106548ad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:33:20.755387 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd47945e-2d03-4a42-afb6-daa106548ad6 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:33:20.755663 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd47945e-2d03-4a42-afb6-daa106548ad6 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:33:20.755929 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd47945e-2d03-4a42-afb6-daa106548ad6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:33:20.820527 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd47945e-2d03-4a42-afb6-daa106548ad6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:33:20.821987 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd47945e-2d03-4a42-afb6-daa106548ad6 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:33:20.852488 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:33:20.923080 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 611/1198] 10.222.0.89 () {58 vars in 1282 bytes} [Thu Jul 28 17:33:20 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 17:33:20.928986 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f7c6ce-e1e6-4917-a5a6-6c5c3ab90cd0 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:33:21.026128 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f7c6ce-e1e6-4917-a5a6-6c5c3ab90cd0 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:33:21.028274 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:33:21.062014 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d6f9fd-edcc-4715-b777-8b2d7e97ab3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:33:21.062653 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d6f9fd-edcc-4715-b777-8b2d7e97ab3b None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:33:21.062935 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d6f9fd-edcc-4715-b777-8b2d7e97ab3b None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:33:21.063207 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d6f9fd-edcc-4715-b777-8b2d7e97ab3b None admin] PATH_INFO: `/v3/projects/12c5f40927a5460ca4193afa70b5edc6` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:33:21.064351 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d6f9fd-edcc-4715-b777-8b2d7e97ab3b None admin] RBAC: Authorizing `identity:get_project(project_id=12c5f40927a5460ca4193afa70b5edc6)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:33:21.065802 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d6f9fd-edcc-4715-b777-8b2d7e97ab3b None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:33:21.067457 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 588/1199] 10.222.0.89 () {56 vars in 1186 bytes} [Thu Jul 28 17:33:20 2022] GET /identity/v3/projects/12c5f40927a5460ca4193afa70b5edc6 => generated 361 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:33:21.073709 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd47945e-2d03-4a42-afb6-daa106548ad6 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:33:21.172360 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd47945e-2d03-4a42-afb6-daa106548ad6 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:33:21.201053 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:33:21.207598 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a223b72c-52a7-4980-9227-730bf22b082e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:33:21.208202 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a223b72c-52a7-4980-9227-730bf22b082e None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:33:21.208479 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a223b72c-52a7-4980-9227-730bf22b082e None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:33:21.208751 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a223b72c-52a7-4980-9227-730bf22b082e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:33:21.209554 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a223b72c-52a7-4980-9227-730bf22b082e None admin] RBAC: Adding query filter params (role.id=73e457231a824ea98c28d22610be640c, scope.project.id=12c5f40927a5460ca4193afa70b5edc6, user.id=1aefea8c2dcb4b22bf732f2ebb0c8fe2) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:33:21.209863 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a223b72c-52a7-4980-9227-730bf22b082e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:33:21.211295 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a223b72c-52a7-4980-9227-730bf22b082e None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:33:21.217558 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 612/1200] 10.222.0.89 () {56 vars in 1374 bytes} [Thu Jul 28 17:33:21 2022] GET /identity/v3/role_assignments?user.id=1aefea8c2dcb4b22bf732f2ebb0c8fe2&scope.project.id=12c5f40927a5460ca4193afa70b5edc6&role.id=73e457231a824ea98c28d22610be640c => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:33:31.057692 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=14 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:33:31.214576 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=2 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:34:16.788068 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc62c24-c93c-4bc9-b3e5-83f1ec832eea None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:34:16.788672 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc62c24-c93c-4bc9-b3e5-83f1ec832eea None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:34:16.788672 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc62c24-c93c-4bc9-b3e5-83f1ec832eea None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:34:16.790434 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 589/1201] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:34:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 17:34:16.801254 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b82b0f-e118-4c91-89c8-95d5cbb1bdf7 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:34:16.801254 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b82b0f-e118-4c91-89c8-95d5cbb1bdf7 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:34:16.801254 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b82b0f-e118-4c91-89c8-95d5cbb1bdf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:34:16.822265 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-d0b82b0f-e118-4c91-89c8-95d5cbb1bdf7 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:34:16.824559 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-d0b82b0f-e118-4c91-89c8-95d5cbb1bdf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:34:16.917058 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 613/1202] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:34:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:34:16.946010 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc62c24-c93c-4bc9-b3e5-83f1ec832eea None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:34:17.070681 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc62c24-c93c-4bc9-b3e5-83f1ec832eea None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:34:17.111531 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9135e4c-57a1-4333-9999-a643c95a5a83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:34:17.112255 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9135e4c-57a1-4333-9999-a643c95a5a83 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:34:17.112542 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9135e4c-57a1-4333-9999-a643c95a5a83 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:34:17.112814 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9135e4c-57a1-4333-9999-a643c95a5a83 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:34:17.129141 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9135e4c-57a1-4333-9999-a643c95a5a83 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:34:17.130753 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9135e4c-57a1-4333-9999-a643c95a5a83 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:34:17.149594 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-f9135e4c-57a1-4333-9999-a643c95a5a83 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:34:17.149594 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:34:17.149594 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:34:17.149594 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:34:17.149594 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:34:17.149594 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:34:17.149594 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:34:17.149594 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:34:17.149594 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:34:17.149594 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:34:17.149594 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:34:17.149594 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:34:17.149594 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:34:17.149594 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:34:17.149594 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:34:17.150669 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:34:17.150669 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:34:17.150669 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:34:17.150669 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:34:17.150669 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:34:17.150669 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:34:17.150669 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:34:17.150669 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:34:17.150669 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:34:17.150669 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:34:17.150669 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:34:17.150669 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:34:17.150669 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:34:17.150669 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:34:17.150669 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:34:17.150669 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:34:17.150669 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:34:17.151810 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:34:17.151810 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:34:17.151810 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:34:17.151810 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:34:17.151810 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 590/1203] 10.222.0.89 () {56 vars in 1108 bytes} [Thu Jul 28 17:34:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:34:17.157859 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b82b0f-e118-4c91-89c8-95d5cbb1bdf7 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:34:17.287714 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b82b0f-e118-4c91-89c8-95d5cbb1bdf7 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:34:17.324669 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:34:17.330989 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c5d2c3-09cd-484f-a13f-ed49992c75e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:34:17.334933 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c5d2c3-09cd-484f-a13f-ed49992c75e1 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:34:17.335270 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c5d2c3-09cd-484f-a13f-ed49992c75e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:34:17.335614 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c5d2c3-09cd-484f-a13f-ed49992c75e1 None admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:34:17.336457 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c5d2c3-09cd-484f-a13f-ed49992c75e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:34:17.337081 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c5d2c3-09cd-484f-a13f-ed49992c75e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:34:17.339116 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c5d2c3-09cd-484f-a13f-ed49992c75e1 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:34:17.362065 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 614/1204] 10.222.0.89 () {56 vars in 1109 bytes} [Thu Jul 28 17:34:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:34:17.362274 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9135e4c-57a1-4333-9999-a643c95a5a83 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:34:17.425418 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:34:17.486155 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9135e4c-57a1-4333-9999-a643c95a5a83 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:34:17.555391 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7c16f6-138e-473a-a92e-35673175e240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:34:17.556379 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7c16f6-138e-473a-a92e-35673175e240 None admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:34:17.556665 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7c16f6-138e-473a-a92e-35673175e240 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:34:17.556942 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7c16f6-138e-473a-a92e-35673175e240 None admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:34:17.557692 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7c16f6-138e-473a-a92e-35673175e240 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:34:17.559481 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7c16f6-138e-473a-a92e-35673175e240 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:34:17.592584 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 591/1205] 10.222.0.89 () {60 vars in 1132 bytes} [Thu Jul 28 17:34:17 2022] POST /identity/v3/users => generated 251 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 17:34:19.957471 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4575a2-5bea-4a95-ada4-a2b0914ceebd None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:34:19.958128 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4575a2-5bea-4a95-ada4-a2b0914ceebd None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:34:19.958128 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4575a2-5bea-4a95-ada4-a2b0914ceebd None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:34:19.959524 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 615/1206] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:34:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:34:19.965440 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb4a4bf-9088-4083-83a3-575516bdde23 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:34:19.965933 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb4a4bf-9088-4083-83a3-575516bdde23 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:34:19.966021 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb4a4bf-9088-4083-83a3-575516bdde23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:34:19.987566 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-5eb4a4bf-9088-4083-83a3-575516bdde23 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:34:19.989044 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-5eb4a4bf-9088-4083-83a3-575516bdde23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:34:20.067385 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 592/1207] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:34:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 17:34:20.092215 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4575a2-5bea-4a95-ada4-a2b0914ceebd None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:34:20.222030 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4575a2-5bea-4a95-ada4-a2b0914ceebd None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:34:20.267986 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83d369d-b167-44c4-82da-f97038e988d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:34:20.269132 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83d369d-b167-44c4-82da-f97038e988d8 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:34:20.269629 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83d369d-b167-44c4-82da-f97038e988d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:34:20.270154 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83d369d-b167-44c4-82da-f97038e988d8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:34:20.286082 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83d369d-b167-44c4-82da-f97038e988d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:34:20.291971 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-a83d369d-b167-44c4-82da-f97038e988d8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 17:34:20.291971 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:34:20.291971 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:34:20.291971 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:34:20.291971 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:34:20.291971 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:34:20.291971 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:34:20.291971 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:34:20.291971 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:34:20.291971 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:34:20.291971 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:34:20.291971 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:34:20.291971 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:34:20.291971 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:34:20.291971 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:34:20.291971 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:34:20.294223 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:34:20.294223 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:34:20.294223 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:34:20.294223 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:34:20.294223 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:34:20.294223 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:34:20.294223 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:34:20.294223 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:34:20.294223 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:34:20.294223 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:34:20.294223 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:34:20.294223 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( Jul 28 17:34:20.294223 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:34:20.294223 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:34:20.294223 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:34:20.294223 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:34:20.295087 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:34:20.295087 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:34:20.295087 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:34:20.295087 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:34:20.295087 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:34:20.295087 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:34:20.295087 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:34:20.295087 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:34:20.295087 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:34:20.295087 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:34:20.295087 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 17:34:20.295087 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:34:20.295087 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 616/1208] 10.222.0.89 () {56 vars in 1102 bytes} [Thu Jul 28 17:34:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:34:20.300559 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb4a4bf-9088-4083-83a3-575516bdde23 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:34:20.347862 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:34:20.416446 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb4a4bf-9088-4083-83a3-575516bdde23 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:34:20.454871 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f758b4-c964-43e8-9eb2-3fea78733b31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:34:20.455371 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f758b4-c964-43e8-9eb2-3fea78733b31 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:34:20.455656 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f758b4-c964-43e8-9eb2-3fea78733b31 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:34:20.455929 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f758b4-c964-43e8-9eb2-3fea78733b31 None admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:34:20.456653 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f758b4-c964-43e8-9eb2-3fea78733b31 None admin] RBAC: Adding query filter params (name=service) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:34:20.456961 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f758b4-c964-43e8-9eb2-3fea78733b31 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:34:20.458368 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f758b4-c964-43e8-9eb2-3fea78733b31 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:34:20.468855 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 593/1209] 10.222.0.89 () {56 vars in 1103 bytes} [Thu Jul 28 17:34:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:34:20.475307 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83d369d-b167-44c4-82da-f97038e988d8 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:34:20.537246 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:34:20.616648 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83d369d-b167-44c4-82da-f97038e988d8 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:34:20.669376 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864b265c-1c01-4cc2-86fd-eaf76f4ed8fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:34:20.670107 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864b265c-1c01-4cc2-86fd-eaf76f4ed8fc None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:34:20.670415 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864b265c-1c01-4cc2-86fd-eaf76f4ed8fc None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:34:20.671706 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864b265c-1c01-4cc2-86fd-eaf76f4ed8fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:34:20.687876 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864b265c-1c01-4cc2-86fd-eaf76f4ed8fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:34:20.689603 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864b265c-1c01-4cc2-86fd-eaf76f4ed8fc None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:34:20.709655 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-864b265c-1c01-4cc2-86fd-eaf76f4ed8fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:34:20.709655 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:34:20.709655 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:34:20.709655 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:34:20.709655 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:34:20.709655 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:34:20.709655 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:34:20.709655 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:34:20.709655 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:34:20.709655 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:34:20.709655 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:34:20.709655 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:34:20.709655 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:34:20.709655 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:34:20.709655 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:34:20.711030 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:34:20.711030 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:34:20.711030 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:34:20.711030 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:34:20.711030 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:34:20.711030 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:34:20.711030 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:34:20.711030 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:34:20.711030 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( Jul 28 17:34:20.711030 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:34:20.711030 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:34:20.711030 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:34:20.711030 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:34:20.711030 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:34:20.711030 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:34:20.711030 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:34:20.711030 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:34:20.716668 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:34:20.716668 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:34:20.716668 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:34:20.716668 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:34:20.716668 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 617/1210] 10.222.0.89 () {56 vars in 1108 bytes} [Thu Jul 28 17:34:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:34:20.716970 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f758b4-c964-43e8-9eb2-3fea78733b31 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:34:20.837429 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f758b4-c964-43e8-9eb2-3fea78733b31 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:34:20.839300 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:34:20.888920 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed98aa1a-a69d-4ea4-b6bd-9559f43c8b46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:34:20.890629 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed98aa1a-a69d-4ea4-b6bd-9559f43c8b46 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:34:20.890981 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed98aa1a-a69d-4ea4-b6bd-9559f43c8b46 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:34:20.891251 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed98aa1a-a69d-4ea4-b6bd-9559f43c8b46 None admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:34:20.891964 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed98aa1a-a69d-4ea4-b6bd-9559f43c8b46 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:34:20.892402 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed98aa1a-a69d-4ea4-b6bd-9559f43c8b46 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:34:20.894186 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed98aa1a-a69d-4ea4-b6bd-9559f43c8b46 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:34:20.910839 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 594/1211] 10.222.0.89 () {56 vars in 1109 bytes} [Thu Jul 28 17:34:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:34:20.919838 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864b265c-1c01-4cc2-86fd-eaf76f4ed8fc None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:34:21.057993 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864b265c-1c01-4cc2-86fd-eaf76f4ed8fc None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:34:21.059550 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:34:21.104807 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08eff2f4-6d03-49f3-b1c2-cfd59aa626fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:34:21.105549 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eff2f4-6d03-49f3-b1c2-cfd59aa626fc None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:34:21.105871 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eff2f4-6d03-49f3-b1c2-cfd59aa626fc None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:34:21.106155 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eff2f4-6d03-49f3-b1c2-cfd59aa626fc None admin] PATH_INFO: `/v3/users/placement` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:34:21.119186 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08eff2f4-6d03-49f3-b1c2-cfd59aa626fc None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:34:21.120938 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08eff2f4-6d03-49f3-b1c2-cfd59aa626fc None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:34:21.137344 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-08eff2f4-6d03-49f3-b1c2-cfd59aa626fc None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 28 17:34:21.137344 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:34:21.137344 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:34:21.137344 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:34:21.137344 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:34:21.137344 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:34:21.137344 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:34:21.137344 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:34:21.137344 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:34:21.137344 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:34:21.137344 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:34:21.137344 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:34:21.137344 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 17:34:21.137344 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 17:34:21.137344 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 17:34:21.137344 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 17:34:21.138595 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:34:21.138595 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:34:21.138595 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 17:34:21.138595 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:34:21.138595 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 17:34:21.138595 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:34:21.138595 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:34:21.138595 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:34:21.138595 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:34:21.138595 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:34:21.138595 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:34:21.138595 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( Jul 28 17:34:21.138595 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:34:21.138595 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:34:21.138595 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:34:21.138595 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:34:21.138595 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:34:21.138595 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:34:21.140206 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:34:21.140206 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:34:21.140206 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 17:34:21.140206 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 17:34:21.140206 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 17:34:21.140206 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 17:34:21.140206 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 17:34:21.140206 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 17:34:21.140206 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 28 17:34:21.140206 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:34:21.140206 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 618/1212] 10.222.0.89 () {56 vars in 1108 bytes} [Thu Jul 28 17:34:20 2022] GET /identity/v3/users/placement => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:34:21.146257 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed98aa1a-a69d-4ea4-b6bd-9559f43c8b46 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:34:21.283413 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed98aa1a-a69d-4ea4-b6bd-9559f43c8b46 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:34:21.328956 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a35b1ce-50c9-4ee4-9513-9b4841ada52c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:34:21.329747 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a35b1ce-50c9-4ee4-9513-9b4841ada52c None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:34:21.330026 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a35b1ce-50c9-4ee4-9513-9b4841ada52c None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:34:21.330296 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a35b1ce-50c9-4ee4-9513-9b4841ada52c None admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:34:21.331094 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a35b1ce-50c9-4ee4-9513-9b4841ada52c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:34:21.331396 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a35b1ce-50c9-4ee4-9513-9b4841ada52c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:34:21.332825 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a35b1ce-50c9-4ee4-9513-9b4841ada52c None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:34:21.355342 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 595/1213] 10.222.0.89 () {56 vars in 1143 bytes} [Thu Jul 28 17:34:21 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:34:21.362265 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08eff2f4-6d03-49f3-b1c2-cfd59aa626fc None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:34:21.494414 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08eff2f4-6d03-49f3-b1c2-cfd59aa626fc None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:34:21.538299 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44a1d2b-1ac2-49c8-8441-882dfcfca885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:34:21.538922 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44a1d2b-1ac2-49c8-8441-882dfcfca885 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:34:21.539215 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44a1d2b-1ac2-49c8-8441-882dfcfca885 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:34:21.539474 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44a1d2b-1ac2-49c8-8441-882dfcfca885 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:34:21.585387 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:34:21.634174 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44a1d2b-1ac2-49c8-8441-882dfcfca885 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:34:21.635800 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44a1d2b-1ac2-49c8-8441-882dfcfca885 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:34:21.763580 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 619/1214] 10.222.0.89 () {58 vars in 1282 bytes} [Thu Jul 28 17:34:21 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 28 17:34:21.769816 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a35b1ce-50c9-4ee4-9513-9b4841ada52c None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:34:21.789521 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:34:21.899237 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a35b1ce-50c9-4ee4-9513-9b4841ada52c None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:34:21.943153 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2d681c-c211-426d-a173-f8280d403c54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:34:21.943871 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2d681c-c211-426d-a173-f8280d403c54 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:34:21.944194 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2d681c-c211-426d-a173-f8280d403c54 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:34:21.944480 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2d681c-c211-426d-a173-f8280d403c54 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:34:21.957848 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2d681c-c211-426d-a173-f8280d403c54 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:34:21.959371 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2d681c-c211-426d-a173-f8280d403c54 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:34:21.976666 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-bd2d681c-c211-426d-a173-f8280d403c54 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:34:21.976666 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:34:21.976666 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:34:21.976666 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:34:21.976666 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:34:21.976666 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:34:21.976666 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:34:21.976666 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:34:21.976666 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:34:21.976666 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:34:21.976666 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:34:21.976666 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:34:21.976666 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:34:21.976666 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:34:21.976666 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:34:21.977714 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:34:21.977714 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:34:21.977714 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:34:21.977714 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:34:21.977714 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:34:21.977714 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:34:21.977714 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:34:21.977714 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:34:21.977714 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:34:21.977714 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:34:21.977714 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:34:21.977714 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:34:21.977714 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:34:21.977714 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:34:21.977714 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:34:21.977714 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:34:21.977714 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:34:21.978626 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:34:21.978626 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:34:21.978626 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:34:21.978626 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:34:21.978626 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 596/1215] 10.222.0.89 () {56 vars in 1108 bytes} [Thu Jul 28 17:34:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:34:21.984010 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44a1d2b-1ac2-49c8-8441-882dfcfca885 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:34:22.052219 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:34:22.102016 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44a1d2b-1ac2-49c8-8441-882dfcfca885 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:34:22.146240 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5ada7f-3668-47db-9b4a-d00ac1ae5d73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:34:22.146876 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5ada7f-3668-47db-9b4a-d00ac1ae5d73 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:34:22.147158 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5ada7f-3668-47db-9b4a-d00ac1ae5d73 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:34:22.147434 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5ada7f-3668-47db-9b4a-d00ac1ae5d73 None admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:34:22.148127 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5ada7f-3668-47db-9b4a-d00ac1ae5d73 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:34:22.148432 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5ada7f-3668-47db-9b4a-d00ac1ae5d73 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:34:22.149825 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5ada7f-3668-47db-9b4a-d00ac1ae5d73 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:34:22.165442 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 620/1216] 10.222.0.89 () {56 vars in 1109 bytes} [Thu Jul 28 17:34:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:34:22.171468 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2d681c-c211-426d-a173-f8280d403c54 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:34:22.255705 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:34:22.290718 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2d681c-c211-426d-a173-f8280d403c54 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:34:22.331630 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc977192-194d-4423-8019-8718a0293017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:34:22.332178 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc977192-194d-4423-8019-8718a0293017 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:34:22.332454 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc977192-194d-4423-8019-8718a0293017 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:34:22.332722 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc977192-194d-4423-8019-8718a0293017 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:34:22.340206 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc977192-194d-4423-8019-8718a0293017 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:34:22.341761 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc977192-194d-4423-8019-8718a0293017 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:34:22.356636 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-dc977192-194d-4423-8019-8718a0293017 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:34:22.356636 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:34:22.356636 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:34:22.356636 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:34:22.356636 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:34:22.356636 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:34:22.356636 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:34:22.356636 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:34:22.356636 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:34:22.356636 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:34:22.356636 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:34:22.356636 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:34:22.356636 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 17:34:22.356636 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 17:34:22.356636 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 17:34:22.357841 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 17:34:22.357841 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:34:22.357841 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:34:22.357841 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:34:22.357841 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:34:22.357841 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:34:22.357841 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:34:22.357841 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:34:22.357841 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:34:22.357841 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:34:22.357841 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:34:22.357841 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:34:22.357841 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:34:22.357841 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:34:22.357841 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:34:22.357841 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:34:22.357841 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:34:22.359527 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 17:34:22.359527 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 17:34:22.359527 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 17:34:22.359527 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 17:34:22.359527 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 17:34:22.359527 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 17:34:22.359527 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:34:22.359527 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:34:22.361642 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 597/1217] 10.222.0.89 () {56 vars in 1111 bytes} [Thu Jul 28 17:34:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:34:22.367752 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5ada7f-3668-47db-9b4a-d00ac1ae5d73 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:34:22.487330 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5ada7f-3668-47db-9b4a-d00ac1ae5d73 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:34:22.512264 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:34:22.529265 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e5e903-2292-4268-931f-fd3628676441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:34:22.530474 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e5e903-2292-4268-931f-fd3628676441 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:34:22.530802 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e5e903-2292-4268-931f-fd3628676441 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:34:22.531124 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e5e903-2292-4268-931f-fd3628676441 None admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:34:22.531990 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e5e903-2292-4268-931f-fd3628676441 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:34:22.532313 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e5e903-2292-4268-931f-fd3628676441 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:34:22.533834 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e5e903-2292-4268-931f-fd3628676441 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:34:22.548613 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 621/1218] 10.222.0.89 () {56 vars in 1148 bytes} [Thu Jul 28 17:34:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:34:22.554657 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc977192-194d-4423-8019-8718a0293017 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:34:22.677203 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc977192-194d-4423-8019-8718a0293017 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:34:22.708148 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:34:22.714931 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e034b79-4cb0-4918-be63-444156be61a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:34:22.715556 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e034b79-4cb0-4918-be63-444156be61a2 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:34:22.715840 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e034b79-4cb0-4918-be63-444156be61a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:34:22.716116 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e034b79-4cb0-4918-be63-444156be61a2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:34:22.716904 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e034b79-4cb0-4918-be63-444156be61a2 None admin] RBAC: Adding query filter params (role.id=0a9ea6e6d6f94955a5dc949b0746a3f1, scope.project.id=04434e8535a64044996316f1d17dc0bc, user.id=e4bd290018394b0d955cd376ce35b77f) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:34:22.717248 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e034b79-4cb0-4918-be63-444156be61a2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:34:22.719552 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e034b79-4cb0-4918-be63-444156be61a2 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:34:22.726458 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 598/1219] 10.222.0.89 () {56 vars in 1374 bytes} [Thu Jul 28 17:34:22 2022] GET /identity/v3/role_assignments?user.id=e4bd290018394b0d955cd376ce35b77f&scope.project.id=04434e8535a64044996316f1d17dc0bc&role.id=0a9ea6e6d6f94955a5dc949b0746a3f1 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:34:24.966780 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bb5b8e-af4d-4dfc-b08d-1319902c2399 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:34:24.968083 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bb5b8e-af4d-4dfc-b08d-1319902c2399 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:34:24.968083 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bb5b8e-af4d-4dfc-b08d-1319902c2399 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:34:24.969471 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 622/1220] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:34:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 17:34:24.975491 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6d526a-bbea-4854-b857-0db35de6e446 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:34:24.976016 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6d526a-bbea-4854-b857-0db35de6e446 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:34:24.976116 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6d526a-bbea-4854-b857-0db35de6e446 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:34:24.995018 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-6f6d526a-bbea-4854-b857-0db35de6e446 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:34:24.996274 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-6f6d526a-bbea-4854-b857-0db35de6e446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:34:25.070310 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 599/1221] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:34:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:34:25.095955 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bb5b8e-af4d-4dfc-b08d-1319902c2399 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:34:25.206579 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bb5b8e-af4d-4dfc-b08d-1319902c2399 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:34:25.242411 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde95539-6149-45ee-b33d-777e54d7108a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:34:25.243046 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde95539-6149-45ee-b33d-777e54d7108a None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:34:25.243324 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde95539-6149-45ee-b33d-777e54d7108a None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:34:25.243592 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde95539-6149-45ee-b33d-777e54d7108a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:34:25.254457 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde95539-6149-45ee-b33d-777e54d7108a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:34:25.259380 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-fde95539-6149-45ee-b33d-777e54d7108a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 17:34:25.259380 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:34:25.259380 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:34:25.259380 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:34:25.259380 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:34:25.259380 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:34:25.259380 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:34:25.259380 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:34:25.259380 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:34:25.259380 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:34:25.259380 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:34:25.259380 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:34:25.259380 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:34:25.259380 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:34:25.259380 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:34:25.259380 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:34:25.260570 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:34:25.260570 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:34:25.260570 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:34:25.260570 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:34:25.260570 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:34:25.260570 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:34:25.260570 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:34:25.260570 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:34:25.260570 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:34:25.260570 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:34:25.260570 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:34:25.260570 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( Jul 28 17:34:25.260570 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:34:25.260570 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:34:25.260570 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:34:25.260570 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:34:25.261474 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:34:25.261474 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:34:25.261474 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:34:25.261474 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:34:25.261474 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:34:25.261474 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:34:25.261474 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:34:25.261474 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:34:25.261474 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:34:25.261474 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:34:25.261474 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 17:34:25.261474 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:34:25.261474 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 623/1222] 10.222.0.89 () {56 vars in 1102 bytes} [Thu Jul 28 17:34:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:34:25.267523 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6d526a-bbea-4854-b857-0db35de6e446 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:34:25.370974 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6d526a-bbea-4854-b857-0db35de6e446 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:34:25.396772 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:34:25.408188 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ae1483-091a-4feb-ab06-948f8863f259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:34:25.408809 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ae1483-091a-4feb-ab06-948f8863f259 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:34:25.409088 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ae1483-091a-4feb-ab06-948f8863f259 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:34:25.409396 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ae1483-091a-4feb-ab06-948f8863f259 None admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:34:25.410105 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ae1483-091a-4feb-ab06-948f8863f259 None admin] RBAC: Adding query filter params (name=service) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:34:25.410403 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ae1483-091a-4feb-ab06-948f8863f259 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:34:25.411757 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ae1483-091a-4feb-ab06-948f8863f259 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:34:25.420120 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 600/1223] 10.222.0.89 () {56 vars in 1103 bytes} [Thu Jul 28 17:34:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:34:25.426539 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde95539-6149-45ee-b33d-777e54d7108a None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:34:25.474873 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:34:25.538626 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde95539-6149-45ee-b33d-777e54d7108a None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:34:25.575011 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e965d7-674f-4968-a124-d80ca3e435c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:34:25.575658 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e965d7-674f-4968-a124-d80ca3e435c3 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:34:25.575980 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e965d7-674f-4968-a124-d80ca3e435c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:34:25.576257 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e965d7-674f-4968-a124-d80ca3e435c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:34:25.588608 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e965d7-674f-4968-a124-d80ca3e435c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:34:25.590216 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e965d7-674f-4968-a124-d80ca3e435c3 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:34:25.606602 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-b6e965d7-674f-4968-a124-d80ca3e435c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:34:25.606602 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:34:25.606602 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:34:25.606602 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:34:25.606602 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:34:25.606602 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:34:25.606602 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:34:25.606602 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:34:25.606602 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:34:25.606602 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:34:25.606602 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:34:25.606602 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:34:25.606602 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:34:25.606602 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:34:25.606602 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:34:25.607771 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:34:25.607771 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:34:25.607771 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:34:25.607771 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:34:25.607771 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:34:25.607771 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:34:25.607771 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:34:25.607771 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:34:25.607771 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( Jul 28 17:34:25.607771 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:34:25.607771 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:34:25.607771 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:34:25.607771 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:34:25.607771 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:34:25.607771 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:34:25.607771 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:34:25.607771 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:34:25.613662 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:34:25.613662 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:34:25.613662 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:34:25.613662 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:34:25.613662 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 624/1224] 10.222.0.89 () {56 vars in 1108 bytes} [Thu Jul 28 17:34:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:34:25.614025 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ae1483-091a-4feb-ab06-948f8863f259 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:34:25.720220 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ae1483-091a-4feb-ab06-948f8863f259 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:34:25.755610 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59264cc2-051c-4538-b382-d793707b63ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:34:25.756252 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59264cc2-051c-4538-b382-d793707b63ee None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:34:25.756536 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59264cc2-051c-4538-b382-d793707b63ee None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:34:25.756805 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59264cc2-051c-4538-b382-d793707b63ee None admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:34:25.757546 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59264cc2-051c-4538-b382-d793707b63ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:34:25.757887 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59264cc2-051c-4538-b382-d793707b63ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:34:25.759358 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59264cc2-051c-4538-b382-d793707b63ee None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:34:25.772236 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 601/1225] 10.222.0.89 () {56 vars in 1109 bytes} [Thu Jul 28 17:34:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:34:25.778255 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e965d7-674f-4968-a124-d80ca3e435c3 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:34:25.894700 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:34:25.902775 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e965d7-674f-4968-a124-d80ca3e435c3 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:34:25.948824 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14759e0f-ec67-4a9d-9d87-22d783f15b66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:34:25.950062 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14759e0f-ec67-4a9d-9d87-22d783f15b66 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:34:25.950593 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14759e0f-ec67-4a9d-9d87-22d783f15b66 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:34:25.950994 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14759e0f-ec67-4a9d-9d87-22d783f15b66 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:34:25.961906 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14759e0f-ec67-4a9d-9d87-22d783f15b66 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:34:25.963376 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14759e0f-ec67-4a9d-9d87-22d783f15b66 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:34:25.976601 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-14759e0f-ec67-4a9d-9d87-22d783f15b66 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 28 17:34:25.976601 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:34:25.976601 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:34:25.976601 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:34:25.976601 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:34:25.976601 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:34:25.976601 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:34:25.976601 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:34:25.976601 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:34:25.976601 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:34:25.976601 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:34:25.976601 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:34:25.976601 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 17:34:25.976601 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 17:34:25.976601 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 17:34:25.976601 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 17:34:25.978159 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:34:25.978159 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:34:25.978159 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 17:34:25.978159 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:34:25.978159 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 17:34:25.978159 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:34:25.978159 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:34:25.978159 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:34:25.978159 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:34:25.978159 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:34:25.978159 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:34:25.978159 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( Jul 28 17:34:25.978159 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:34:25.978159 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:34:25.978159 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:34:25.978159 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:34:25.978159 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:34:25.978159 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:34:25.979276 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:34:25.979276 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:34:25.979276 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 17:34:25.979276 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 17:34:25.979276 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 17:34:25.979276 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 17:34:25.979276 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 17:34:25.979276 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 17:34:25.979276 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 28 17:34:25.979276 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:34:25.979276 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 625/1226] 10.222.0.89 () {56 vars in 1108 bytes} [Thu Jul 28 17:34:25 2022] GET /identity/v3/users/placement => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:34:25.984534 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59264cc2-051c-4538-b382-d793707b63ee None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:34:26.038473 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:34:26.102154 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59264cc2-051c-4538-b382-d793707b63ee None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:34:26.143832 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f6bf30-5d77-465c-a5e2-d514a0487599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:34:26.144574 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f6bf30-5d77-465c-a5e2-d514a0487599 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:34:26.144905 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f6bf30-5d77-465c-a5e2-d514a0487599 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:34:26.145235 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f6bf30-5d77-465c-a5e2-d514a0487599 None admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:34:26.145982 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f6bf30-5d77-465c-a5e2-d514a0487599 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:34:26.146459 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f6bf30-5d77-465c-a5e2-d514a0487599 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:34:26.147873 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f6bf30-5d77-465c-a5e2-d514a0487599 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:34:26.166640 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 602/1227] 10.222.0.89 () {56 vars in 1143 bytes} [Thu Jul 28 17:34:25 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:34:26.172962 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14759e0f-ec67-4a9d-9d87-22d783f15b66 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:34:26.283651 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14759e0f-ec67-4a9d-9d87-22d783f15b66 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:34:26.323515 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bdc99e-a182-45e5-8369-143d79921e67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:34:26.324157 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bdc99e-a182-45e5-8369-143d79921e67 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:34:26.324436 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bdc99e-a182-45e5-8369-143d79921e67 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:34:26.324706 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bdc99e-a182-45e5-8369-143d79921e67 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:34:26.338067 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bdc99e-a182-45e5-8369-143d79921e67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:34:26.339718 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bdc99e-a182-45e5-8369-143d79921e67 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:34:26.356307 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-c4bdc99e-a182-45e5-8369-143d79921e67 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:34:26.356307 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:34:26.356307 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:34:26.356307 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:34:26.356307 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:34:26.356307 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:34:26.356307 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:34:26.356307 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:34:26.356307 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:34:26.356307 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:34:26.356307 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:34:26.356307 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:34:26.356307 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:34:26.356307 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:34:26.356307 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:34:26.357571 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:34:26.357571 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:34:26.357571 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:34:26.357571 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:34:26.357571 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:34:26.357571 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:34:26.357571 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:34:26.357571 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:34:26.357571 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( Jul 28 17:34:26.357571 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:34:26.357571 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:34:26.357571 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:34:26.357571 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:34:26.357571 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:34:26.357571 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:34:26.357571 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:34:26.357571 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:34:26.359205 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:34:26.359205 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:34:26.359205 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:34:26.359205 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:34:26.359205 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 626/1228] 10.222.0.89 () {56 vars in 1108 bytes} [Thu Jul 28 17:34:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:34:26.363965 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f6bf30-5d77-465c-a5e2-d514a0487599 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:34:26.480196 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f6bf30-5d77-465c-a5e2-d514a0487599 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:34:26.483306 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:34:26.524612 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5ea237-1f6d-4f9a-aa6c-2c46a00959a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:34:26.525307 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5ea237-1f6d-4f9a-aa6c-2c46a00959a8 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:34:26.527535 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5ea237-1f6d-4f9a-aa6c-2c46a00959a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:34:26.527535 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5ea237-1f6d-4f9a-aa6c-2c46a00959a8 None admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:34:26.527535 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5ea237-1f6d-4f9a-aa6c-2c46a00959a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:34:26.527535 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5ea237-1f6d-4f9a-aa6c-2c46a00959a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:34:26.528324 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5ea237-1f6d-4f9a-aa6c-2c46a00959a8 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:34:26.543266 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 603/1229] 10.222.0.89 () {56 vars in 1109 bytes} [Thu Jul 28 17:34:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:34:26.549826 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bdc99e-a182-45e5-8369-143d79921e67 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:34:26.563126 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:34:26.684630 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bdc99e-a182-45e5-8369-143d79921e67 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:34:26.724985 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fe8a27-3079-4ff2-b99b-905dddc62c27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:34:26.725961 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fe8a27-3079-4ff2-b99b-905dddc62c27 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:34:26.726390 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fe8a27-3079-4ff2-b99b-905dddc62c27 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:34:26.726672 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fe8a27-3079-4ff2-b99b-905dddc62c27 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:34:26.735632 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fe8a27-3079-4ff2-b99b-905dddc62c27 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:34:26.737265 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fe8a27-3079-4ff2-b99b-905dddc62c27 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:34:26.753480 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-f1fe8a27-3079-4ff2-b99b-905dddc62c27 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:34:26.753480 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:34:26.753480 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:34:26.753480 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:34:26.753480 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:34:26.753480 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:34:26.753480 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:34:26.753480 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:34:26.753480 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:34:26.753480 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:34:26.753480 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:34:26.753480 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:34:26.753480 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 17:34:26.753480 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 17:34:26.753480 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 17:34:26.754562 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 17:34:26.754562 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:34:26.754562 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:34:26.754562 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:34:26.754562 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:34:26.754562 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:34:26.754562 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:34:26.754562 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:34:26.754562 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( Jul 28 17:34:26.754562 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:34:26.754562 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:34:26.754562 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:34:26.754562 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:34:26.754562 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:34:26.754562 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:34:26.754562 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:34:26.754562 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:34:26.755526 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 17:34:26.755526 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 17:34:26.755526 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 17:34:26.755526 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 17:34:26.755526 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 17:34:26.755526 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 17:34:26.755526 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:34:26.755526 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:34:26.755526 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 627/1230] 10.222.0.89 () {56 vars in 1111 bytes} [Thu Jul 28 17:34:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:34:26.762723 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5ea237-1f6d-4f9a-aa6c-2c46a00959a8 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:34:26.883729 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5ea237-1f6d-4f9a-aa6c-2c46a00959a8 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:34:26.917561 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a87967-8b11-4839-b6a7-75fc808c6d99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:34:26.918233 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a87967-8b11-4839-b6a7-75fc808c6d99 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:34:26.918483 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a87967-8b11-4839-b6a7-75fc808c6d99 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:34:26.918733 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a87967-8b11-4839-b6a7-75fc808c6d99 None admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:34:26.919490 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a87967-8b11-4839-b6a7-75fc808c6d99 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:34:26.919802 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a87967-8b11-4839-b6a7-75fc808c6d99 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:34:26.921881 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a87967-8b11-4839-b6a7-75fc808c6d99 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:34:26.934758 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 604/1231] 10.222.0.89 () {56 vars in 1148 bytes} [Thu Jul 28 17:34:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:34:26.941432 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fe8a27-3079-4ff2-b99b-905dddc62c27 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:34:27.013580 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:34:27.057952 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fe8a27-3079-4ff2-b99b-905dddc62c27 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:34:27.100935 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657b4e93-c6b6-458f-9253-c0f835176227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:34:27.100935 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657b4e93-c6b6-458f-9253-c0f835176227 None admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:34:27.100935 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657b4e93-c6b6-458f-9253-c0f835176227 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:34:27.101369 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657b4e93-c6b6-458f-9253-c0f835176227 None admin] PATH_INFO: `/v3/projects/04434e8535a64044996316f1d17dc0bc/users/e4bd290018394b0d955cd376ce35b77f/roles/0a9ea6e6d6f94955a5dc949b0746a3f1` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:34:27.117256 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657b4e93-c6b6-458f-9253-c0f835176227 None admin] RBAC: Authorizing `identity:create_grant(project_id=04434e8535a64044996316f1d17dc0bc, user_id=e4bd290018394b0d955cd376ce35b77f, role_id=0a9ea6e6d6f94955a5dc949b0746a3f1)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:34:27.118733 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657b4e93-c6b6-458f-9253-c0f835176227 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:34:27.132672 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 628/1232] 10.222.0.89 () {58 vars in 1439 bytes} [Thu Jul 28 17:34:26 2022] PUT /identity/v3/projects/04434e8535a64044996316f1d17dc0bc/users/e4bd290018394b0d955cd376ce35b77f/roles/0a9ea6e6d6f94955a5dc949b0746a3f1 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:34:29.263655 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2205c8d8-0203-45ba-9bc2-1425db10baa5 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:34:29.264213 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2205c8d8-0203-45ba-9bc2-1425db10baa5 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:34:29.264213 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2205c8d8-0203-45ba-9bc2-1425db10baa5 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:34:29.266121 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 605/1233] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:34:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:34:29.271329 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb481321-9724-4849-86c5-23dfc41d482a None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:34:29.271760 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb481321-9724-4849-86c5-23dfc41d482a None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:34:29.271889 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb481321-9724-4849-86c5-23dfc41d482a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:34:29.289706 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-bb481321-9724-4849-86c5-23dfc41d482a None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:34:29.290902 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-bb481321-9724-4849-86c5-23dfc41d482a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:34:29.366334 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 629/1234] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:34:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:34:29.390815 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2205c8d8-0203-45ba-9bc2-1425db10baa5 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:34:29.417953 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:34:29.499567 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2205c8d8-0203-45ba-9bc2-1425db10baa5 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:34:29.538899 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c816b18f-f7d6-4ff9-aa8e-603502185ebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:34:29.539379 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c816b18f-f7d6-4ff9-aa8e-603502185ebf None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:34:29.539685 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c816b18f-f7d6-4ff9-aa8e-603502185ebf None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:34:29.539932 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c816b18f-f7d6-4ff9-aa8e-603502185ebf None admin] PATH_INFO: `/v3/roles/service` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:34:29.552053 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c816b18f-f7d6-4ff9-aa8e-603502185ebf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:34:29.557104 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-c816b18f-f7d6-4ff9-aa8e-603502185ebf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 17:34:29.557104 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:34:29.557104 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:34:29.557104 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:34:29.557104 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:34:29.557104 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:34:29.557104 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:34:29.557104 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:34:29.557104 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:34:29.557104 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:34:29.557104 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:34:29.557104 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:34:29.557104 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:34:29.557104 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:34:29.557104 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:34:29.557104 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:34:29.558809 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:34:29.558809 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:34:29.558809 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:34:29.558809 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:34:29.558809 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:34:29.558809 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:34:29.558809 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:34:29.558809 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:34:29.558809 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:34:29.558809 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:34:29.558809 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:34:29.558809 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:34:29.558809 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:34:29.558809 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:34:29.558809 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:34:29.558809 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:34:29.560125 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:34:29.560125 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:34:29.560125 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:34:29.560125 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:34:29.560125 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:34:29.560125 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:34:29.560125 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:34:29.560125 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:34:29.560125 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:34:29.560125 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:34:29.560125 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 17:34:29.560125 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:34:29.560125 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 606/1235] 10.222.0.89 () {56 vars in 1102 bytes} [Thu Jul 28 17:34:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:34:29.566527 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb481321-9724-4849-86c5-23dfc41d482a None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:34:29.616193 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:34:29.677562 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb481321-9724-4849-86c5-23dfc41d482a None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:34:29.715025 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534f4865-34de-4bbf-85f9-c173f47d93a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:34:29.715705 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534f4865-34de-4bbf-85f9-c173f47d93a6 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:34:29.716013 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534f4865-34de-4bbf-85f9-c173f47d93a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:34:29.716313 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534f4865-34de-4bbf-85f9-c173f47d93a6 None admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:34:29.717097 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534f4865-34de-4bbf-85f9-c173f47d93a6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:34:29.718008 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534f4865-34de-4bbf-85f9-c173f47d93a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:34:29.719695 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534f4865-34de-4bbf-85f9-c173f47d93a6 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:34:29.728412 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 630/1236] 10.222.0.89 () {56 vars in 1103 bytes} [Thu Jul 28 17:34:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:34:29.735189 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c816b18f-f7d6-4ff9-aa8e-603502185ebf None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:34:29.831769 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:34:29.849558 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c816b18f-f7d6-4ff9-aa8e-603502185ebf None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:34:29.886722 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d9a467-d45e-45f3-ad9b-0d2b8c8a36e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:34:29.887363 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d9a467-d45e-45f3-ad9b-0d2b8c8a36e6 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:34:29.887641 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d9a467-d45e-45f3-ad9b-0d2b8c8a36e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:34:29.887911 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d9a467-d45e-45f3-ad9b-0d2b8c8a36e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:34:29.899973 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d9a467-d45e-45f3-ad9b-0d2b8c8a36e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:34:29.901503 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d9a467-d45e-45f3-ad9b-0d2b8c8a36e6 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:34:29.917397 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-27d9a467-d45e-45f3-ad9b-0d2b8c8a36e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:34:29.917397 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:34:29.917397 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:34:29.917397 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:34:29.917397 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:34:29.917397 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:34:29.917397 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:34:29.917397 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:34:29.917397 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:34:29.917397 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:34:29.917397 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:34:29.917397 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:34:29.917397 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:34:29.917397 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:34:29.917397 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:34:29.918537 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:34:29.918537 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:34:29.918537 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:34:29.918537 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:34:29.918537 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:34:29.918537 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:34:29.918537 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:34:29.918537 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:34:29.918537 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:34:29.918537 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:34:29.918537 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:34:29.918537 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:34:29.918537 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:34:29.918537 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:34:29.918537 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:34:29.918537 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:34:29.918537 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:34:29.919748 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:34:29.919748 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:34:29.919748 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:34:29.919748 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:34:29.921968 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 607/1237] 10.222.0.89 () {56 vars in 1108 bytes} [Thu Jul 28 17:34:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:34:29.928813 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534f4865-34de-4bbf-85f9-c173f47d93a6 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:34:30.039375 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534f4865-34de-4bbf-85f9-c173f47d93a6 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:34:30.049890 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:34:30.080181 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2e58ec-b921-4cfa-bc02-10f956e99cbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:34:30.080710 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2e58ec-b921-4cfa-bc02-10f956e99cbc None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:34:30.080995 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2e58ec-b921-4cfa-bc02-10f956e99cbc None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:34:30.081309 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2e58ec-b921-4cfa-bc02-10f956e99cbc None admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:34:30.082041 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2e58ec-b921-4cfa-bc02-10f956e99cbc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:34:30.082317 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2e58ec-b921-4cfa-bc02-10f956e99cbc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:34:30.083738 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2e58ec-b921-4cfa-bc02-10f956e99cbc None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:34:30.097971 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 631/1238] 10.222.0.89 () {56 vars in 1109 bytes} [Thu Jul 28 17:34:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:34:30.103978 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d9a467-d45e-45f3-ad9b-0d2b8c8a36e6 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:34:30.214038 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d9a467-d45e-45f3-ad9b-0d2b8c8a36e6 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:34:30.250999 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4398c005-1a5f-4042-a40e-f881eaf72514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:34:30.251643 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4398c005-1a5f-4042-a40e-f881eaf72514 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:34:30.251922 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4398c005-1a5f-4042-a40e-f881eaf72514 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:34:30.252195 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4398c005-1a5f-4042-a40e-f881eaf72514 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:34:30.254981 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:34:30.260867 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4398c005-1a5f-4042-a40e-f881eaf72514 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:34:30.262573 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4398c005-1a5f-4042-a40e-f881eaf72514 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:34:30.274369 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-4398c005-1a5f-4042-a40e-f881eaf72514 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 28 17:34:30.274369 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:34:30.274369 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:34:30.274369 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:34:30.274369 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:34:30.274369 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:34:30.274369 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:34:30.274369 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:34:30.274369 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:34:30.274369 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:34:30.274369 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:34:30.274369 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:34:30.274369 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 17:34:30.274369 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 17:34:30.274369 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 17:34:30.274369 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 17:34:30.276094 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:34:30.276094 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:34:30.276094 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 17:34:30.276094 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:34:30.276094 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 17:34:30.276094 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:34:30.276094 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:34:30.276094 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:34:30.276094 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:34:30.276094 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:34:30.276094 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:34:30.276094 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:34:30.276094 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:34:30.276094 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:34:30.276094 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:34:30.276094 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:34:30.276094 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:34:30.276094 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:34:30.277358 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:34:30.277358 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:34:30.277358 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 17:34:30.277358 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 17:34:30.277358 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 17:34:30.277358 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 17:34:30.277358 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 17:34:30.277358 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 17:34:30.277358 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 28 17:34:30.277358 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:34:30.277358 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 608/1239] 10.222.0.89 () {56 vars in 1108 bytes} [Thu Jul 28 17:34:30 2022] GET /identity/v3/users/placement => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:34:30.282055 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2e58ec-b921-4cfa-bc02-10f956e99cbc None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:34:30.402115 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2e58ec-b921-4cfa-bc02-10f956e99cbc None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:34:30.440108 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2554006b-83ba-4173-9016-23f52b7b4f14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:34:30.440724 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2554006b-83ba-4173-9016-23f52b7b4f14 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:34:30.441032 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2554006b-83ba-4173-9016-23f52b7b4f14 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:34:30.441321 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2554006b-83ba-4173-9016-23f52b7b4f14 None admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:34:30.442151 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2554006b-83ba-4173-9016-23f52b7b4f14 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:34:30.442465 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2554006b-83ba-4173-9016-23f52b7b4f14 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:34:30.443919 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2554006b-83ba-4173-9016-23f52b7b4f14 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:34:30.462747 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 632/1240] 10.222.0.89 () {56 vars in 1143 bytes} [Thu Jul 28 17:34:30 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:34:30.469339 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4398c005-1a5f-4042-a40e-f881eaf72514 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:34:30.600697 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4398c005-1a5f-4042-a40e-f881eaf72514 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:34:30.639366 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08ac1ec-fd74-40c8-9936-26b46e33104d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:34:30.640794 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08ac1ec-fd74-40c8-9936-26b46e33104d None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:34:30.641118 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08ac1ec-fd74-40c8-9936-26b46e33104d None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:34:30.641506 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08ac1ec-fd74-40c8-9936-26b46e33104d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:34:30.713490 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:34:30.722090 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08ac1ec-fd74-40c8-9936-26b46e33104d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:34:30.724228 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08ac1ec-fd74-40c8-9936-26b46e33104d None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:34:30.833475 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 609/1241] 10.222.0.89 () {58 vars in 1282 bytes} [Thu Jul 28 17:34:30 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 28 17:34:30.839253 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2554006b-83ba-4173-9016-23f52b7b4f14 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:34:30.869373 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:34:30.954021 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2554006b-83ba-4173-9016-23f52b7b4f14 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:34:30.992759 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6920cb-ebe7-4591-9ad7-bfb316a4c138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:34:30.993273 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6920cb-ebe7-4591-9ad7-bfb316a4c138 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:34:30.993556 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6920cb-ebe7-4591-9ad7-bfb316a4c138 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:34:30.993827 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6920cb-ebe7-4591-9ad7-bfb316a4c138 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:34:31.006314 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6920cb-ebe7-4591-9ad7-bfb316a4c138 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:34:31.007908 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6920cb-ebe7-4591-9ad7-bfb316a4c138 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:34:31.024712 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-ec6920cb-ebe7-4591-9ad7-bfb316a4c138 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:34:31.024712 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:34:31.024712 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:34:31.024712 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:34:31.024712 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:34:31.024712 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:34:31.024712 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:34:31.024712 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:34:31.024712 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:34:31.024712 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:34:31.024712 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:34:31.024712 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:34:31.024712 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:34:31.024712 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:34:31.024712 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:34:31.025720 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:34:31.025720 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:34:31.025720 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:34:31.025720 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:34:31.025720 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:34:31.025720 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:34:31.025720 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:34:31.025720 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:34:31.025720 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( Jul 28 17:34:31.025720 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:34:31.025720 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:34:31.025720 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:34:31.025720 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:34:31.025720 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:34:31.025720 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:34:31.025720 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:34:31.025720 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:34:31.026630 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:34:31.026630 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:34:31.026630 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:34:31.026630 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:34:31.026630 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 633/1242] 10.222.0.89 () {56 vars in 1108 bytes} [Thu Jul 28 17:34:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 17:34:31.032631 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08ac1ec-fd74-40c8-9936-26b46e33104d None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:34:31.147678 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:34:31.154683 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08ac1ec-fd74-40c8-9936-26b46e33104d None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:34:31.192568 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18369035-f816-4de5-962c-e6c354e22b68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:34:31.193205 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18369035-f816-4de5-962c-e6c354e22b68 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:34:31.193494 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18369035-f816-4de5-962c-e6c354e22b68 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:34:31.193803 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18369035-f816-4de5-962c-e6c354e22b68 None admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:34:31.194468 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18369035-f816-4de5-962c-e6c354e22b68 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:34:31.194776 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18369035-f816-4de5-962c-e6c354e22b68 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:34:31.196152 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18369035-f816-4de5-962c-e6c354e22b68 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:34:31.209508 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 610/1243] 10.222.0.89 () {56 vars in 1109 bytes} [Thu Jul 28 17:34:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:34:31.215494 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6920cb-ebe7-4591-9ad7-bfb316a4c138 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:34:31.302381 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:34:31.325417 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6920cb-ebe7-4591-9ad7-bfb316a4c138 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:34:31.370006 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f07f6f-e911-404e-aa47-ec733896998d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:34:31.370811 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f07f6f-e911-404e-aa47-ec733896998d None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:34:31.371182 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f07f6f-e911-404e-aa47-ec733896998d None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:34:31.371777 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f07f6f-e911-404e-aa47-ec733896998d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:34:31.381972 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f07f6f-e911-404e-aa47-ec733896998d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:34:31.384005 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f07f6f-e911-404e-aa47-ec733896998d None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:34:31.397622 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-37f07f6f-e911-404e-aa47-ec733896998d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:34:31.397622 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:34:31.397622 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:34:31.397622 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:34:31.397622 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:34:31.397622 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:34:31.397622 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:34:31.397622 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:34:31.397622 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:34:31.397622 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:34:31.397622 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:34:31.397622 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:34:31.397622 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 17:34:31.397622 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 17:34:31.397622 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 17:34:31.398763 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 17:34:31.398763 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:34:31.398763 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:34:31.398763 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:34:31.398763 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:34:31.398763 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:34:31.398763 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:34:31.398763 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:34:31.398763 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( Jul 28 17:34:31.398763 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:34:31.398763 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:34:31.398763 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:34:31.398763 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:34:31.398763 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:34:31.398763 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:34:31.398763 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:34:31.398763 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:34:31.399824 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 17:34:31.399824 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 17:34:31.399824 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 17:34:31.399824 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 17:34:31.399824 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 17:34:31.399824 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 17:34:31.399824 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:34:31.399824 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:34:31.399824 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 634/1244] 10.222.0.89 () {56 vars in 1111 bytes} [Thu Jul 28 17:34:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:34:31.404798 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18369035-f816-4de5-962c-e6c354e22b68 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:34:31.514805 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18369035-f816-4de5-962c-e6c354e22b68 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:34:31.554837 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefd59ac-1abe-40d1-8404-9cffd03f0cd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:34:31.555564 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefd59ac-1abe-40d1-8404-9cffd03f0cd6 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:34:31.555869 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefd59ac-1abe-40d1-8404-9cffd03f0cd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:34:31.556206 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefd59ac-1abe-40d1-8404-9cffd03f0cd6 None admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:34:31.557211 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefd59ac-1abe-40d1-8404-9cffd03f0cd6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:34:31.557725 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefd59ac-1abe-40d1-8404-9cffd03f0cd6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:34:31.559518 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefd59ac-1abe-40d1-8404-9cffd03f0cd6 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:34:31.573012 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 611/1245] 10.222.0.89 () {56 vars in 1148 bytes} [Thu Jul 28 17:34:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:34:31.578934 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f07f6f-e911-404e-aa47-ec733896998d None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:34:31.694293 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f07f6f-e911-404e-aa47-ec733896998d None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:34:31.736738 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6893458d-b948-4805-89c6-5957ec8fb614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:34:31.737601 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6893458d-b948-4805-89c6-5957ec8fb614 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:34:31.738165 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6893458d-b948-4805-89c6-5957ec8fb614 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:34:31.738512 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6893458d-b948-4805-89c6-5957ec8fb614 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:34:31.739671 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6893458d-b948-4805-89c6-5957ec8fb614 None admin] RBAC: Adding query filter params (role.id=0a9ea6e6d6f94955a5dc949b0746a3f1, scope.project.id=04434e8535a64044996316f1d17dc0bc, user.id=e4bd290018394b0d955cd376ce35b77f) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:34:31.740227 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6893458d-b948-4805-89c6-5957ec8fb614 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:34:31.741898 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6893458d-b948-4805-89c6-5957ec8fb614 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:34:31.750529 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 635/1246] 10.222.0.89 () {56 vars in 1374 bytes} [Thu Jul 28 17:34:31 2022] GET /identity/v3/role_assignments?user.id=e4bd290018394b0d955cd376ce35b77f&scope.project.id=04434e8535a64044996316f1d17dc0bc&role.id=0a9ea6e6d6f94955a5dc949b0746a3f1 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:34:33.916475 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcaf103-dbde-40f4-adce-2b0226a0ece9 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:34:33.917064 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcaf103-dbde-40f4-adce-2b0226a0ece9 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:34:33.917220 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcaf103-dbde-40f4-adce-2b0226a0ece9 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:34:33.918745 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 612/1247] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:34:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:34:33.925107 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdf398c-e917-4ab5-b671-42d53e540aaf None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:34:33.925614 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdf398c-e917-4ab5-b671-42d53e540aaf None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:34:33.925780 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdf398c-e917-4ab5-b671-42d53e540aaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:34:33.932101 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:34:33.945632 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-bcdf398c-e917-4ab5-b671-42d53e540aaf None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:34:33.946997 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-bcdf398c-e917-4ab5-b671-42d53e540aaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:34:34.027426 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 636/1248] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:34:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:34:34.053276 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcaf103-dbde-40f4-adce-2b0226a0ece9 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:34:34.075984 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:34:34.173276 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcaf103-dbde-40f4-adce-2b0226a0ece9 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:34:34.209352 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad2b235-743a-4a77-95ef-462cbb4e53ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:34:34.209997 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad2b235-743a-4a77-95ef-462cbb4e53ca None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:34:34.210412 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad2b235-743a-4a77-95ef-462cbb4e53ca None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:34:34.210536 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad2b235-743a-4a77-95ef-462cbb4e53ca None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:34:34.223302 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad2b235-743a-4a77-95ef-462cbb4e53ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:34:34.232528 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-9ad2b235-743a-4a77-95ef-462cbb4e53ca None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:34:34.232528 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:34:34.232528 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:34:34.232528 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:34:34.232528 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:34:34.232528 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:34:34.232528 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:34:34.232528 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:34:34.232528 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:34:34.232528 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:34:34.232528 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:34:34.232528 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:34:34.232528 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:34:34.232528 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:34:34.232528 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:34:34.232528 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:34:34.235811 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:34:34.235811 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:34:34.235811 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:34:34.235811 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:34:34.235811 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:34:34.235811 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:34:34.235811 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:34:34.235811 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:34:34.235811 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:34:34.235811 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:34:34.235811 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:34:34.235811 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:34:34.235811 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:34:34.235811 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:34:34.235811 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:34:34.235811 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:34:34.237546 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:34:34.237546 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:34:34.237546 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:34:34.237546 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:34:34.237546 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:34:34.237546 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:34:34.237546 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:34:34.237546 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:34:34.237546 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:34:34.237546 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:34:34.237546 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:34:34.237546 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:34:34.237546 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 613/1249] 10.222.0.89 () {56 vars in 1096 bytes} [Thu Jul 28 17:34:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:34:34.243120 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdf398c-e917-4ab5-b671-42d53e540aaf None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:34:34.360567 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdf398c-e917-4ab5-b671-42d53e540aaf None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:34:34.396815 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:34:34.402060 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635ac006-11a4-44f4-91e1-1b5068ee4b7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:34:34.402690 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635ac006-11a4-44f4-91e1-1b5068ee4b7d None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:34:34.403125 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635ac006-11a4-44f4-91e1-1b5068ee4b7d None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:34:34.403564 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635ac006-11a4-44f4-91e1-1b5068ee4b7d None admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:34:34.404274 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635ac006-11a4-44f4-91e1-1b5068ee4b7d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:34:34.404676 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635ac006-11a4-44f4-91e1-1b5068ee4b7d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:34:34.406455 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635ac006-11a4-44f4-91e1-1b5068ee4b7d None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:34:34.421229 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 637/1250] 10.222.0.89 () {56 vars in 1099 bytes} [Thu Jul 28 17:34:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:34:34.423626 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad2b235-743a-4a77-95ef-462cbb4e53ca None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:34:34.518580 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:34:34.546215 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad2b235-743a-4a77-95ef-462cbb4e53ca None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:34:34.585527 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24d8288-8e7d-469f-865d-6fe805778ae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:34:34.586299 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24d8288-8e7d-469f-865d-6fe805778ae9 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:34:34.586558 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24d8288-8e7d-469f-865d-6fe805778ae9 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:34:34.586864 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24d8288-8e7d-469f-865d-6fe805778ae9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:34:34.598654 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24d8288-8e7d-469f-865d-6fe805778ae9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:34:34.600161 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24d8288-8e7d-469f-865d-6fe805778ae9 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:34:34.616600 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-d24d8288-8e7d-469f-865d-6fe805778ae9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:34:34.616600 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:34:34.616600 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:34:34.616600 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:34:34.616600 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:34:34.616600 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:34:34.616600 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:34:34.616600 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:34:34.616600 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:34:34.616600 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:34:34.616600 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:34:34.616600 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:34:34.616600 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:34:34.616600 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:34:34.616600 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:34:34.617668 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:34:34.617668 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:34:34.617668 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:34:34.617668 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:34:34.617668 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:34:34.617668 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:34:34.617668 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:34:34.617668 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:34:34.617668 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:34:34.617668 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:34:34.617668 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:34:34.617668 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:34:34.617668 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:34:34.617668 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:34:34.617668 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:34:34.617668 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:34:34.617668 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:34:34.618576 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:34:34.618576 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:34:34.618576 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:34:34.618576 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:34:34.619024 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 614/1251] 10.222.0.89 () {56 vars in 1108 bytes} [Thu Jul 28 17:34:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:34:34.626413 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635ac006-11a4-44f4-91e1-1b5068ee4b7d None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:34:34.737687 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635ac006-11a4-44f4-91e1-1b5068ee4b7d None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:34:34.775719 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c497d45e-f4da-453f-9b7c-66151fe3d604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:34:34.776347 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c497d45e-f4da-453f-9b7c-66151fe3d604 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:34:34.776632 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c497d45e-f4da-453f-9b7c-66151fe3d604 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:34:34.776917 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c497d45e-f4da-453f-9b7c-66151fe3d604 None admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:34:34.777755 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c497d45e-f4da-453f-9b7c-66151fe3d604 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:34:34.778063 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c497d45e-f4da-453f-9b7c-66151fe3d604 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:34:34.779445 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c497d45e-f4da-453f-9b7c-66151fe3d604 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:34:34.792756 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 638/1252] 10.222.0.89 () {56 vars in 1109 bytes} [Thu Jul 28 17:34:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:34:34.799827 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24d8288-8e7d-469f-865d-6fe805778ae9 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:34:34.918598 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24d8288-8e7d-469f-865d-6fe805778ae9 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:34:34.946286 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:34:34.955842 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbe87f0-e990-4a71-9b33-8338295d0821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:34:34.956492 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbe87f0-e990-4a71-9b33-8338295d0821 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:34:34.956773 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbe87f0-e990-4a71-9b33-8338295d0821 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:34:34.957049 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbe87f0-e990-4a71-9b33-8338295d0821 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:34:34.966077 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbe87f0-e990-4a71-9b33-8338295d0821 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:34:34.967599 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbe87f0-e990-4a71-9b33-8338295d0821 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:34:34.980612 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-9cbe87f0-e990-4a71-9b33-8338295d0821 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 28 17:34:34.980612 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:34:34.980612 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:34:34.980612 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:34:34.980612 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:34:34.980612 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:34:34.980612 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:34:34.980612 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:34:34.980612 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:34:34.980612 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:34:34.980612 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:34:34.980612 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:34:34.980612 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 17:34:34.980612 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 17:34:34.980612 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 17:34:34.980612 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 17:34:34.981915 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:34:34.981915 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:34:34.981915 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 17:34:34.981915 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:34:34.981915 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 17:34:34.981915 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:34:34.981915 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:34:34.981915 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:34:34.981915 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:34:34.981915 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:34:34.981915 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:34:34.981915 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:34:34.981915 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:34:34.981915 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:34:34.981915 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:34:34.981915 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:34:34.981915 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:34:34.981915 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:34:34.983128 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:34:34.983128 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:34:34.983128 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 17:34:34.983128 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 17:34:34.983128 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 17:34:34.983128 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 17:34:34.983128 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 17:34:34.983128 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 17:34:34.983128 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 28 17:34:34.983128 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:34:34.983128 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 615/1253] 10.222.0.89 () {56 vars in 1108 bytes} [Thu Jul 28 17:34:34 2022] GET /identity/v3/users/placement => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:34:34.987672 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c497d45e-f4da-453f-9b7c-66151fe3d604 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:34:35.062482 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:34:35.111763 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c497d45e-f4da-453f-9b7c-66151fe3d604 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:34:35.150363 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99db5f8e-03ea-4f43-8684-04549be635a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:34:35.150977 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99db5f8e-03ea-4f43-8684-04549be635a4 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:34:35.151261 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99db5f8e-03ea-4f43-8684-04549be635a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:34:35.151538 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99db5f8e-03ea-4f43-8684-04549be635a4 None admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:34:35.152337 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99db5f8e-03ea-4f43-8684-04549be635a4 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:34:35.152649 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99db5f8e-03ea-4f43-8684-04549be635a4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:34:35.154181 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99db5f8e-03ea-4f43-8684-04549be635a4 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:34:35.173849 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 639/1254] 10.222.0.89 () {56 vars in 1143 bytes} [Thu Jul 28 17:34:34 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:34:35.179603 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbe87f0-e990-4a71-9b33-8338295d0821 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:34:35.285469 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbe87f0-e990-4a71-9b33-8338295d0821 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:34:35.322345 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323180a1-fb78-4744-b43a-19fbf3ffad9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:34:35.322896 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323180a1-fb78-4744-b43a-19fbf3ffad9d None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:34:35.323177 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323180a1-fb78-4744-b43a-19fbf3ffad9d None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:34:35.324690 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323180a1-fb78-4744-b43a-19fbf3ffad9d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:34:35.387430 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:34:35.401439 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323180a1-fb78-4744-b43a-19fbf3ffad9d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:34:35.402956 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323180a1-fb78-4744-b43a-19fbf3ffad9d None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:34:35.508959 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 616/1255] 10.222.0.89 () {58 vars in 1282 bytes} [Thu Jul 28 17:34:35 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 28 17:34:35.515934 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99db5f8e-03ea-4f43-8684-04549be635a4 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:34:35.653908 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99db5f8e-03ea-4f43-8684-04549be635a4 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:34:35.674270 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:34:35.705577 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb141d58-20cf-44d7-ac3a-dea4e49bec36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:34:35.706330 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb141d58-20cf-44d7-ac3a-dea4e49bec36 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:34:35.706618 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb141d58-20cf-44d7-ac3a-dea4e49bec36 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:34:35.706893 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb141d58-20cf-44d7-ac3a-dea4e49bec36 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:34:35.719034 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb141d58-20cf-44d7-ac3a-dea4e49bec36 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:34:35.720498 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb141d58-20cf-44d7-ac3a-dea4e49bec36 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:34:35.738304 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-eb141d58-20cf-44d7-ac3a-dea4e49bec36 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:34:35.738304 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:34:35.738304 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:34:35.738304 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:34:35.738304 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:34:35.738304 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:34:35.738304 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:34:35.738304 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:34:35.738304 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:34:35.738304 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:34:35.738304 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:34:35.738304 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:34:35.738304 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:34:35.738304 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:34:35.738304 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:34:35.739472 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:34:35.739472 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:34:35.739472 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:34:35.739472 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:34:35.739472 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:34:35.739472 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:34:35.739472 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:34:35.739472 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:34:35.739472 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( Jul 28 17:34:35.739472 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:34:35.739472 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:34:35.739472 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:34:35.739472 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:34:35.739472 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:34:35.739472 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:34:35.739472 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:34:35.739472 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:34:35.740493 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:34:35.740493 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:34:35.740493 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:34:35.740493 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:34:35.740493 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 640/1256] 10.222.0.89 () {56 vars in 1108 bytes} [Thu Jul 28 17:34:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:34:35.748459 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323180a1-fb78-4744-b43a-19fbf3ffad9d None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:34:35.848401 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:34:35.863264 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323180a1-fb78-4744-b43a-19fbf3ffad9d None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:34:35.903261 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb694e4-4721-4e4d-a52b-a695466c5b13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:34:35.904162 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb694e4-4721-4e4d-a52b-a695466c5b13 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:34:35.904451 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb694e4-4721-4e4d-a52b-a695466c5b13 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:34:35.905057 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb694e4-4721-4e4d-a52b-a695466c5b13 None admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:34:35.906460 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb694e4-4721-4e4d-a52b-a695466c5b13 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:34:35.906770 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb694e4-4721-4e4d-a52b-a695466c5b13 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:34:35.908652 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb694e4-4721-4e4d-a52b-a695466c5b13 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:34:35.922729 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 617/1257] 10.222.0.89 () {56 vars in 1109 bytes} [Thu Jul 28 17:34:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:34:35.928502 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb141d58-20cf-44d7-ac3a-dea4e49bec36 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:34:36.039453 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb141d58-20cf-44d7-ac3a-dea4e49bec36 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:34:36.080049 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d2255d-161b-43ec-8dc1-5c4d7743d510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:34:36.080683 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d2255d-161b-43ec-8dc1-5c4d7743d510 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:34:36.080962 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d2255d-161b-43ec-8dc1-5c4d7743d510 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:34:36.081261 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d2255d-161b-43ec-8dc1-5c4d7743d510 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:34:36.088981 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d2255d-161b-43ec-8dc1-5c4d7743d510 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:34:36.090884 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d2255d-161b-43ec-8dc1-5c4d7743d510 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:34:36.104006 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-95d2255d-161b-43ec-8dc1-5c4d7743d510 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:34:36.104006 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:34:36.104006 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:34:36.104006 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:34:36.104006 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:34:36.104006 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:34:36.104006 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:34:36.104006 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:34:36.104006 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:34:36.104006 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:34:36.104006 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:34:36.104006 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:34:36.104006 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 17:34:36.104006 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 17:34:36.104006 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 17:34:36.105084 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 17:34:36.105084 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:34:36.105084 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:34:36.105084 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:34:36.105084 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:34:36.105084 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:34:36.105084 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:34:36.105084 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:34:36.105084 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( Jul 28 17:34:36.105084 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:34:36.105084 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:34:36.105084 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:34:36.105084 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:34:36.105084 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:34:36.105084 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:34:36.105084 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:34:36.105084 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:34:36.106518 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 17:34:36.106518 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 17:34:36.106518 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 17:34:36.106518 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 17:34:36.106518 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 17:34:36.106518 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 17:34:36.106518 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:34:36.106518 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:34:36.106518 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 641/1258] 10.222.0.89 () {56 vars in 1111 bytes} [Thu Jul 28 17:34:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:34:36.112560 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb694e4-4721-4e4d-a52b-a695466c5b13 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:34:36.231422 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb694e4-4721-4e4d-a52b-a695466c5b13 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:34:36.271790 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23ed44f-6878-4484-9ae3-60c0b010f0cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:34:36.272489 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23ed44f-6878-4484-9ae3-60c0b010f0cd None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:34:36.272859 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23ed44f-6878-4484-9ae3-60c0b010f0cd None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:34:36.273144 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23ed44f-6878-4484-9ae3-60c0b010f0cd None admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:34:36.274036 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23ed44f-6878-4484-9ae3-60c0b010f0cd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:34:36.274318 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23ed44f-6878-4484-9ae3-60c0b010f0cd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:34:36.275947 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23ed44f-6878-4484-9ae3-60c0b010f0cd None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:34:36.289382 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 618/1259] 10.222.0.89 () {56 vars in 1148 bytes} [Thu Jul 28 17:34:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:34:36.295544 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d2255d-161b-43ec-8dc1-5c4d7743d510 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:34:36.323284 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:34:36.412828 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d2255d-161b-43ec-8dc1-5c4d7743d510 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:34:36.449685 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c184ea6d-1e5d-496a-857f-4396f2ae8fb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:34:36.450423 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c184ea6d-1e5d-496a-857f-4396f2ae8fb0 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:34:36.450708 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c184ea6d-1e5d-496a-857f-4396f2ae8fb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:34:36.450991 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c184ea6d-1e5d-496a-857f-4396f2ae8fb0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:34:36.452817 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c184ea6d-1e5d-496a-857f-4396f2ae8fb0 None admin] RBAC: Adding query filter params (role.id=73e457231a824ea98c28d22610be640c, scope.project.id=04434e8535a64044996316f1d17dc0bc, user.id=e4bd290018394b0d955cd376ce35b77f) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:34:36.453126 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c184ea6d-1e5d-496a-857f-4396f2ae8fb0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:34:36.455629 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c184ea6d-1e5d-496a-857f-4396f2ae8fb0 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:34:36.462872 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 642/1260] 10.222.0.89 () {56 vars in 1374 bytes} [Thu Jul 28 17:34:36 2022] GET /identity/v3/role_assignments?user.id=e4bd290018394b0d955cd376ce35b77f&scope.project.id=04434e8535a64044996316f1d17dc0bc&role.id=73e457231a824ea98c28d22610be640c => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:34:38.489988 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5abee28-2685-42df-bda5-6a859a54d2d0 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:34:38.490522 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5abee28-2685-42df-bda5-6a859a54d2d0 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:34:38.490522 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5abee28-2685-42df-bda5-6a859a54d2d0 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:34:38.491889 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 619/1261] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:34:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:34:38.497031 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e8f604-d18c-4ad5-9aa5-99c2c7c91a8b None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:34:38.497459 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e8f604-d18c-4ad5-9aa5-99c2c7c91a8b None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:34:38.497616 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e8f604-d18c-4ad5-9aa5-99c2c7c91a8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:34:38.518031 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-c0e8f604-d18c-4ad5-9aa5-99c2c7c91a8b None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:34:38.519263 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-c0e8f604-d18c-4ad5-9aa5-99c2c7c91a8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:34:38.589238 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 643/1262] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:34:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:34:38.611753 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5abee28-2685-42df-bda5-6a859a54d2d0 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:34:38.622450 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:34:38.712199 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5abee28-2685-42df-bda5-6a859a54d2d0 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:34:38.745620 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de667bb5-5db4-4e60-b866-8eca4d99c141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:34:38.746188 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de667bb5-5db4-4e60-b866-8eca4d99c141 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:34:38.746463 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de667bb5-5db4-4e60-b866-8eca4d99c141 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:34:38.746729 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de667bb5-5db4-4e60-b866-8eca4d99c141 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:34:38.757030 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de667bb5-5db4-4e60-b866-8eca4d99c141 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:34:38.761890 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-de667bb5-5db4-4e60-b866-8eca4d99c141 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:34:38.761890 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:34:38.761890 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:34:38.761890 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:34:38.761890 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:34:38.761890 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:34:38.761890 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:34:38.761890 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:34:38.761890 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:34:38.761890 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:34:38.761890 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:34:38.761890 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:34:38.761890 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:34:38.761890 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:34:38.761890 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:34:38.761890 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:34:38.762978 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:34:38.762978 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:34:38.762978 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:34:38.762978 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:34:38.762978 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:34:38.762978 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:34:38.762978 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:34:38.762978 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:34:38.762978 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:34:38.762978 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:34:38.762978 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:34:38.762978 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:34:38.762978 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:34:38.762978 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:34:38.762978 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:34:38.762978 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:34:38.764019 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:34:38.764019 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:34:38.764019 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:34:38.764019 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:34:38.764019 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:34:38.764019 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:34:38.764019 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:34:38.764019 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:34:38.764019 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:34:38.764019 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:34:38.764019 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:34:38.764019 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:34:38.764019 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 620/1263] 10.222.0.89 () {56 vars in 1096 bytes} [Thu Jul 28 17:34:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:34:38.769616 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e8f604-d18c-4ad5-9aa5-99c2c7c91a8b None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:34:38.788073 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:34:38.868484 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e8f604-d18c-4ad5-9aa5-99c2c7c91a8b None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:34:38.901948 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a98058-581a-4f19-bd11-214edad35bc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:34:38.902547 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a98058-581a-4f19-bd11-214edad35bc0 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:34:38.902823 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a98058-581a-4f19-bd11-214edad35bc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:34:38.903092 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a98058-581a-4f19-bd11-214edad35bc0 None admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:34:38.903790 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a98058-581a-4f19-bd11-214edad35bc0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:34:38.904089 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a98058-581a-4f19-bd11-214edad35bc0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:34:38.906458 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a98058-581a-4f19-bd11-214edad35bc0 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:34:38.914277 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 644/1264] 10.222.0.89 () {56 vars in 1099 bytes} [Thu Jul 28 17:34:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:34:38.920032 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de667bb5-5db4-4e60-b866-8eca4d99c141 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:34:38.984985 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:34:39.018497 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de667bb5-5db4-4e60-b866-8eca4d99c141 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:34:39.051521 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5ee132-40c3-45db-a4d8-a72cefc03fea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:34:39.052155 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5ee132-40c3-45db-a4d8-a72cefc03fea None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:34:39.052434 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5ee132-40c3-45db-a4d8-a72cefc03fea None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:34:39.052706 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5ee132-40c3-45db-a4d8-a72cefc03fea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:34:39.062599 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5ee132-40c3-45db-a4d8-a72cefc03fea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:34:39.064043 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5ee132-40c3-45db-a4d8-a72cefc03fea None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:34:39.079879 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-8c5ee132-40c3-45db-a4d8-a72cefc03fea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:34:39.079879 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:34:39.079879 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:34:39.079879 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:34:39.079879 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:34:39.079879 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:34:39.079879 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:34:39.079879 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:34:39.079879 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:34:39.079879 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:34:39.079879 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:34:39.079879 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:34:39.079879 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:34:39.079879 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:34:39.079879 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:34:39.080996 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:34:39.080996 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:34:39.080996 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:34:39.080996 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:34:39.080996 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:34:39.080996 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:34:39.080996 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:34:39.080996 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:34:39.080996 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:34:39.080996 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:34:39.080996 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:34:39.080996 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:34:39.080996 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:34:39.080996 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:34:39.080996 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:34:39.080996 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:34:39.080996 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:34:39.081974 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:34:39.081974 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:34:39.081974 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:34:39.081974 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:34:39.081974 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 621/1265] 10.222.0.89 () {56 vars in 1108 bytes} [Thu Jul 28 17:34:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:34:39.086750 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a98058-581a-4f19-bd11-214edad35bc0 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:34:39.166764 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:34:39.185796 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a98058-581a-4f19-bd11-214edad35bc0 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:34:39.219607 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68886c56-b391-43e6-a1e4-93edb8174c23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:34:39.220234 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68886c56-b391-43e6-a1e4-93edb8174c23 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:34:39.220516 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68886c56-b391-43e6-a1e4-93edb8174c23 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:34:39.220793 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68886c56-b391-43e6-a1e4-93edb8174c23 None admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:34:39.221533 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68886c56-b391-43e6-a1e4-93edb8174c23 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:34:39.221840 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68886c56-b391-43e6-a1e4-93edb8174c23 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:34:39.223239 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68886c56-b391-43e6-a1e4-93edb8174c23 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:34:39.235147 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 645/1266] 10.222.0.89 () {56 vars in 1109 bytes} [Thu Jul 28 17:34:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:34:39.240803 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5ee132-40c3-45db-a4d8-a72cefc03fea None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:34:39.340294 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5ee132-40c3-45db-a4d8-a72cefc03fea None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:34:39.359073 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:34:39.372862 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0159cd4-9907-4a8d-9762-9d4a6d82bfcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:34:39.373535 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0159cd4-9907-4a8d-9762-9d4a6d82bfcc None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:34:39.374315 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0159cd4-9907-4a8d-9762-9d4a6d82bfcc None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:34:39.374808 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0159cd4-9907-4a8d-9762-9d4a6d82bfcc None admin] PATH_INFO: `/v3/users/placement` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:34:39.382575 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0159cd4-9907-4a8d-9762-9d4a6d82bfcc None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:34:39.384006 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0159cd4-9907-4a8d-9762-9d4a6d82bfcc None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:34:39.395728 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-c0159cd4-9907-4a8d-9762-9d4a6d82bfcc None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 28 17:34:39.395728 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:34:39.395728 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:34:39.395728 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:34:39.395728 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:34:39.395728 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:34:39.395728 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:34:39.395728 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:34:39.395728 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:34:39.395728 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:34:39.395728 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:34:39.395728 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:34:39.395728 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 17:34:39.395728 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 17:34:39.395728 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 17:34:39.395728 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 17:34:39.396864 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:34:39.396864 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:34:39.396864 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 17:34:39.396864 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:34:39.396864 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 17:34:39.396864 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:34:39.396864 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:34:39.396864 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:34:39.396864 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:34:39.396864 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:34:39.396864 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:34:39.396864 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:34:39.396864 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:34:39.396864 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:34:39.396864 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:34:39.396864 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:34:39.396864 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:34:39.396864 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:34:39.398077 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:34:39.398077 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:34:39.398077 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 17:34:39.398077 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 17:34:39.398077 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 17:34:39.398077 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 17:34:39.398077 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 17:34:39.398077 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 17:34:39.398077 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 28 17:34:39.398077 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:34:39.398077 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 622/1267] 10.222.0.89 () {56 vars in 1108 bytes} [Thu Jul 28 17:34:39 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:34:39.402644 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68886c56-b391-43e6-a1e4-93edb8174c23 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:34:39.502653 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68886c56-b391-43e6-a1e4-93edb8174c23 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:34:39.543244 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8111e0-3afa-4d67-a773-e4278279ac34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:34:39.543914 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8111e0-3afa-4d67-a773-e4278279ac34 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:34:39.544265 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8111e0-3afa-4d67-a773-e4278279ac34 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:34:39.544506 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8111e0-3afa-4d67-a773-e4278279ac34 None admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:34:39.545322 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8111e0-3afa-4d67-a773-e4278279ac34 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:34:39.545630 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8111e0-3afa-4d67-a773-e4278279ac34 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:34:39.547050 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8111e0-3afa-4d67-a773-e4278279ac34 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:34:39.558392 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:34:39.567690 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 646/1268] 10.222.0.89 () {56 vars in 1143 bytes} [Thu Jul 28 17:34:39 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:34:39.573347 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0159cd4-9907-4a8d-9762-9d4a6d82bfcc None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:34:39.672664 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0159cd4-9907-4a8d-9762-9d4a6d82bfcc None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:34:39.704934 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cab1417-72ee-4965-a275-62d78098d468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:34:39.705572 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cab1417-72ee-4965-a275-62d78098d468 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:34:39.705853 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cab1417-72ee-4965-a275-62d78098d468 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:34:39.706125 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cab1417-72ee-4965-a275-62d78098d468 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:34:39.716273 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cab1417-72ee-4965-a275-62d78098d468 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:34:39.718362 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cab1417-72ee-4965-a275-62d78098d468 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:34:39.732170 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-1cab1417-72ee-4965-a275-62d78098d468 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:34:39.732170 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:34:39.732170 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:34:39.732170 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:34:39.732170 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:34:39.732170 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:34:39.732170 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:34:39.732170 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:34:39.732170 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:34:39.732170 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:34:39.732170 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:34:39.732170 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:34:39.732170 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:34:39.732170 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:34:39.732170 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:34:39.733075 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:34:39.733075 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:34:39.733075 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:34:39.733075 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:34:39.733075 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:34:39.733075 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:34:39.733075 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:34:39.733075 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:34:39.733075 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:34:39.733075 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:34:39.733075 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:34:39.733075 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:34:39.733075 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:34:39.733075 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:34:39.733075 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:34:39.733075 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:34:39.733075 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:34:39.734087 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:34:39.734087 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:34:39.734087 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:34:39.734087 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:34:39.734087 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 623/1269] 10.222.0.89 () {56 vars in 1108 bytes} [Thu Jul 28 17:34:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:34:39.739247 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8111e0-3afa-4d67-a773-e4278279ac34 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:34:39.838669 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8111e0-3afa-4d67-a773-e4278279ac34 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:34:39.873327 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5737aa3-7cd8-4156-89ed-433c8259e030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:34:39.874004 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5737aa3-7cd8-4156-89ed-433c8259e030 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:34:39.874004 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5737aa3-7cd8-4156-89ed-433c8259e030 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:34:39.874614 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5737aa3-7cd8-4156-89ed-433c8259e030 None admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:34:39.875297 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5737aa3-7cd8-4156-89ed-433c8259e030 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:34:39.875626 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5737aa3-7cd8-4156-89ed-433c8259e030 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:34:39.877393 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5737aa3-7cd8-4156-89ed-433c8259e030 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:34:39.888462 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 647/1270] 10.222.0.89 () {56 vars in 1109 bytes} [Thu Jul 28 17:34:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:34:39.894752 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cab1417-72ee-4965-a275-62d78098d468 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:34:39.926123 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:34:39.993569 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cab1417-72ee-4965-a275-62d78098d468 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:34:40.027090 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f5c798-94e5-4ae5-9bb1-db66e31bdcc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:34:40.027726 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f5c798-94e5-4ae5-9bb1-db66e31bdcc6 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:34:40.028006 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f5c798-94e5-4ae5-9bb1-db66e31bdcc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:34:40.028279 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f5c798-94e5-4ae5-9bb1-db66e31bdcc6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:34:40.034939 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f5c798-94e5-4ae5-9bb1-db66e31bdcc6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:34:40.036382 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f5c798-94e5-4ae5-9bb1-db66e31bdcc6 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:34:40.047601 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-84f5c798-94e5-4ae5-9bb1-db66e31bdcc6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:34:40.047601 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:34:40.047601 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:34:40.047601 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:34:40.047601 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:34:40.047601 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:34:40.047601 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:34:40.047601 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:34:40.047601 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:34:40.047601 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:34:40.047601 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:34:40.047601 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:34:40.047601 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 17:34:40.047601 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 17:34:40.047601 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 17:34:40.048661 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 17:34:40.048661 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:34:40.048661 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:34:40.048661 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:34:40.048661 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:34:40.048661 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:34:40.048661 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:34:40.048661 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:34:40.048661 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:34:40.048661 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:34:40.048661 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:34:40.048661 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:34:40.048661 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:34:40.048661 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:34:40.048661 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:34:40.048661 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:34:40.048661 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:34:40.049990 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 17:34:40.049990 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 17:34:40.049990 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 17:34:40.049990 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 17:34:40.049990 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 17:34:40.049990 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 17:34:40.049990 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:34:40.049990 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:34:40.049990 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 624/1271] 10.222.0.89 () {56 vars in 1111 bytes} [Thu Jul 28 17:34:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:34:40.054567 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5737aa3-7cd8-4156-89ed-433c8259e030 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:34:40.111204 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:34:40.153976 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5737aa3-7cd8-4156-89ed-433c8259e030 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:34:40.186115 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f87747-9fb6-4a1f-b575-559b434b0785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:34:40.187611 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f87747-9fb6-4a1f-b575-559b434b0785 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:34:40.187891 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f87747-9fb6-4a1f-b575-559b434b0785 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:34:40.188162 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f87747-9fb6-4a1f-b575-559b434b0785 None admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:34:40.188964 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f87747-9fb6-4a1f-b575-559b434b0785 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:34:40.189321 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f87747-9fb6-4a1f-b575-559b434b0785 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:34:40.190741 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f87747-9fb6-4a1f-b575-559b434b0785 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:34:40.202970 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 648/1272] 10.222.0.89 () {56 vars in 1148 bytes} [Thu Jul 28 17:34:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:34:40.209029 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f5c798-94e5-4ae5-9bb1-db66e31bdcc6 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:34:40.297962 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:34:40.306454 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f5c798-94e5-4ae5-9bb1-db66e31bdcc6 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:34:40.339307 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec5cc7b-3ccf-4943-86a0-dec0b2cf46bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:34:40.339963 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec5cc7b-3ccf-4943-86a0-dec0b2cf46bb None admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:34:40.340240 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec5cc7b-3ccf-4943-86a0-dec0b2cf46bb None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:34:40.340511 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec5cc7b-3ccf-4943-86a0-dec0b2cf46bb None admin] PATH_INFO: `/v3/projects/04434e8535a64044996316f1d17dc0bc/users/e4bd290018394b0d955cd376ce35b77f/roles/73e457231a824ea98c28d22610be640c` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:34:40.342453 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec5cc7b-3ccf-4943-86a0-dec0b2cf46bb None admin] RBAC: Authorizing `identity:create_grant(project_id=04434e8535a64044996316f1d17dc0bc, user_id=e4bd290018394b0d955cd376ce35b77f, role_id=73e457231a824ea98c28d22610be640c)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:34:40.344022 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec5cc7b-3ccf-4943-86a0-dec0b2cf46bb None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:34:40.356418 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 625/1273] 10.222.0.89 () {58 vars in 1439 bytes} [Thu Jul 28 17:34:40 2022] PUT /identity/v3/projects/04434e8535a64044996316f1d17dc0bc/users/e4bd290018394b0d955cd376ce35b77f/roles/73e457231a824ea98c28d22610be640c => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:34:42.330145 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64f4abe-0a51-46d7-8e33-31c09e4bb123 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:34:42.330778 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64f4abe-0a51-46d7-8e33-31c09e4bb123 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:34:42.330778 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64f4abe-0a51-46d7-8e33-31c09e4bb123 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:34:42.332019 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 649/1274] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:34:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:34:42.337135 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96986b52-a10c-47e7-9768-09b3d4bf7243 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:34:42.337590 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96986b52-a10c-47e7-9768-09b3d4bf7243 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:34:42.337702 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96986b52-a10c-47e7-9768-09b3d4bf7243 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:34:42.356940 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-96986b52-a10c-47e7-9768-09b3d4bf7243 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:34:42.358798 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-96986b52-a10c-47e7-9768-09b3d4bf7243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:34:42.423570 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 626/1275] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:34:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:34:42.446544 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64f4abe-0a51-46d7-8e33-31c09e4bb123 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:34:42.556744 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64f4abe-0a51-46d7-8e33-31c09e4bb123 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:34:42.574063 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:34:42.591831 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adde737-84da-42dd-bcd1-4246b023244a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:34:42.592423 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adde737-84da-42dd-bcd1-4246b023244a None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:34:42.593742 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adde737-84da-42dd-bcd1-4246b023244a None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:34:42.594029 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adde737-84da-42dd-bcd1-4246b023244a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:34:42.603676 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adde737-84da-42dd-bcd1-4246b023244a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 17:34:42.608938 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-5adde737-84da-42dd-bcd1-4246b023244a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:34:42.608938 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:34:42.608938 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:34:42.608938 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:34:42.608938 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:34:42.608938 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:34:42.608938 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:34:42.608938 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:34:42.608938 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:34:42.608938 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:34:42.608938 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:34:42.608938 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:34:42.608938 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 17:34:42.608938 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 17:34:42.608938 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 17:34:42.608938 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 17:34:42.610191 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 17:34:42.610191 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 17:34:42.610191 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 17:34:42.610191 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 17:34:42.610191 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:34:42.610191 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:34:42.610191 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:34:42.610191 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:34:42.610191 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:34:42.610191 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:34:42.610191 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:34:42.610191 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( Jul 28 17:34:42.610191 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:34:42.610191 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:34:42.610191 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:34:42.610191 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:34:42.611124 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:34:42.611124 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:34:42.611124 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:34:42.611124 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:34:42.611124 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 17:34:42.611124 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 17:34:42.611124 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 17:34:42.611124 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 17:34:42.611124 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 17:34:42.611124 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 17:34:42.611124 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 17:34:42.611124 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:34:42.611124 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 650/1276] 10.222.0.89 () {56 vars in 1096 bytes} [Thu Jul 28 17:34:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:34:42.616156 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96986b52-a10c-47e7-9768-09b3d4bf7243 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:34:42.694636 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:34:42.713886 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96986b52-a10c-47e7-9768-09b3d4bf7243 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:34:42.747497 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80093929-e148-4a5e-8298-901d74a03f58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:34:42.748109 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80093929-e148-4a5e-8298-901d74a03f58 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:34:42.748387 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80093929-e148-4a5e-8298-901d74a03f58 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:34:42.748656 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80093929-e148-4a5e-8298-901d74a03f58 None admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:34:42.749387 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80093929-e148-4a5e-8298-901d74a03f58 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:34:42.749697 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80093929-e148-4a5e-8298-901d74a03f58 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:34:42.751314 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80093929-e148-4a5e-8298-901d74a03f58 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:34:42.759166 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 627/1277] 10.222.0.89 () {56 vars in 1099 bytes} [Thu Jul 28 17:34:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:34:42.765924 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adde737-84da-42dd-bcd1-4246b023244a None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:34:42.867122 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adde737-84da-42dd-bcd1-4246b023244a None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:34:42.900921 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a0b5e4-ff94-44fa-b033-f261085a0568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:34:42.901576 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a0b5e4-ff94-44fa-b033-f261085a0568 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:34:42.901861 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a0b5e4-ff94-44fa-b033-f261085a0568 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:34:42.902136 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a0b5e4-ff94-44fa-b033-f261085a0568 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:34:42.912423 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a0b5e4-ff94-44fa-b033-f261085a0568 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:34:42.913888 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a0b5e4-ff94-44fa-b033-f261085a0568 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:34:42.928958 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-b7a0b5e4-ff94-44fa-b033-f261085a0568 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:34:42.928958 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:34:42.928958 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:34:42.928958 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:34:42.928958 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:34:42.928958 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:34:42.928958 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:34:42.928958 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:34:42.928958 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:34:42.928958 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:34:42.928958 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:34:42.928958 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:34:42.928958 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:34:42.928958 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:34:42.928958 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:34:42.929986 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:34:42.929986 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:34:42.929986 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:34:42.929986 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:34:42.929986 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:34:42.929986 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:34:42.929986 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:34:42.929986 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:34:42.929986 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( Jul 28 17:34:42.929986 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:34:42.929986 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:34:42.929986 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:34:42.929986 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:34:42.929986 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:34:42.929986 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:34:42.929986 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:34:42.929986 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:34:42.931458 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:34:42.931458 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:34:42.931458 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:34:42.931458 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:34:42.931458 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 651/1278] 10.222.0.89 () {56 vars in 1108 bytes} [Thu Jul 28 17:34:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:34:42.935895 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80093929-e148-4a5e-8298-901d74a03f58 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:34:43.034265 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80093929-e148-4a5e-8298-901d74a03f58 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:34:43.067248 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b912fe7d-a1ef-48ed-9661-9a474e54f922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:34:43.067869 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b912fe7d-a1ef-48ed-9661-9a474e54f922 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:34:43.068147 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b912fe7d-a1ef-48ed-9661-9a474e54f922 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:34:43.068418 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b912fe7d-a1ef-48ed-9661-9a474e54f922 None admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:34:43.069108 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b912fe7d-a1ef-48ed-9661-9a474e54f922 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:34:43.069439 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b912fe7d-a1ef-48ed-9661-9a474e54f922 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:34:43.070798 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b912fe7d-a1ef-48ed-9661-9a474e54f922 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:34:43.082517 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 628/1279] 10.222.0.89 () {56 vars in 1109 bytes} [Thu Jul 28 17:34:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:34:43.088222 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a0b5e4-ff94-44fa-b033-f261085a0568 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:34:43.119505 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:34:43.186598 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a0b5e4-ff94-44fa-b033-f261085a0568 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:34:43.219538 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72c55c7-86b2-4962-a2aa-6a8f9c01e4d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:34:43.220158 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72c55c7-86b2-4962-a2aa-6a8f9c01e4d2 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:34:43.220437 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72c55c7-86b2-4962-a2aa-6a8f9c01e4d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:34:43.220710 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72c55c7-86b2-4962-a2aa-6a8f9c01e4d2 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:34:43.227934 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72c55c7-86b2-4962-a2aa-6a8f9c01e4d2 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:34:43.229390 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72c55c7-86b2-4962-a2aa-6a8f9c01e4d2 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:34:43.240775 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-a72c55c7-86b2-4962-a2aa-6a8f9c01e4d2 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 28 17:34:43.240775 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:34:43.240775 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:34:43.240775 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:34:43.240775 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:34:43.240775 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:34:43.240775 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:34:43.240775 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:34:43.240775 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:34:43.240775 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:34:43.240775 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:34:43.240775 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:34:43.240775 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 17:34:43.240775 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 17:34:43.240775 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 17:34:43.240775 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 17:34:43.241885 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:34:43.241885 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:34:43.241885 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 17:34:43.241885 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:34:43.241885 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 17:34:43.241885 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 17:34:43.241885 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:34:43.241885 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:34:43.241885 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:34:43.241885 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:34:43.241885 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:34:43.241885 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( Jul 28 17:34:43.241885 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:34:43.241885 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:34:43.241885 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:34:43.241885 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:34:43.241885 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:34:43.241885 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:34:43.242853 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:34:43.242853 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:34:43.242853 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 17:34:43.242853 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 17:34:43.242853 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 17:34:43.242853 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 17:34:43.242853 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 17:34:43.242853 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 17:34:43.242853 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 28 17:34:43.242853 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:34:43.242853 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 652/1280] 10.222.0.89 () {56 vars in 1108 bytes} [Thu Jul 28 17:34:43 2022] GET /identity/v3/users/placement => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:34:43.247677 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b912fe7d-a1ef-48ed-9661-9a474e54f922 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:34:43.251694 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:34:43.345049 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b912fe7d-a1ef-48ed-9661-9a474e54f922 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:34:43.378217 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9c0cdc-5d08-4537-b0af-c0605a732b0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:34:43.378787 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9c0cdc-5d08-4537-b0af-c0605a732b0e None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:34:43.379065 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9c0cdc-5d08-4537-b0af-c0605a732b0e None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:34:43.379338 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9c0cdc-5d08-4537-b0af-c0605a732b0e None admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:34:43.380120 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9c0cdc-5d08-4537-b0af-c0605a732b0e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:34:43.380426 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9c0cdc-5d08-4537-b0af-c0605a732b0e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:34:43.381871 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9c0cdc-5d08-4537-b0af-c0605a732b0e None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:34:43.398864 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 629/1281] 10.222.0.89 () {56 vars in 1143 bytes} [Thu Jul 28 17:34:43 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:34:43.404531 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72c55c7-86b2-4962-a2aa-6a8f9c01e4d2 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:34:43.494003 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:34:43.504671 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72c55c7-86b2-4962-a2aa-6a8f9c01e4d2 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:34:43.544684 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f779750c-3c45-4814-bed5-5ab2dbd07450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:34:43.545374 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f779750c-3c45-4814-bed5-5ab2dbd07450 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:34:43.546204 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f779750c-3c45-4814-bed5-5ab2dbd07450 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:34:43.546615 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f779750c-3c45-4814-bed5-5ab2dbd07450 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:34:43.614601 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f779750c-3c45-4814-bed5-5ab2dbd07450 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:34:43.616039 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f779750c-3c45-4814-bed5-5ab2dbd07450 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:34:43.703650 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:34:43.713668 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 653/1282] 10.222.0.89 () {58 vars in 1282 bytes} [Thu Jul 28 17:34:43 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 17:34:43.720028 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9c0cdc-5d08-4537-b0af-c0605a732b0e None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:34:43.780524 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:34:43.820580 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9c0cdc-5d08-4537-b0af-c0605a732b0e None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:34:43.854480 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071ade10-e11c-405c-900d-b5095531df46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:34:43.855118 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071ade10-e11c-405c-900d-b5095531df46 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:34:43.855396 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071ade10-e11c-405c-900d-b5095531df46 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:34:43.855667 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071ade10-e11c-405c-900d-b5095531df46 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:34:43.865498 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071ade10-e11c-405c-900d-b5095531df46 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:34:43.866948 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071ade10-e11c-405c-900d-b5095531df46 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:34:43.884833 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-071ade10-e11c-405c-900d-b5095531df46 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:34:43.884833 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:34:43.884833 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:34:43.884833 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:34:43.884833 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:34:43.884833 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:34:43.884833 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:34:43.884833 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:34:43.884833 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:34:43.884833 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:34:43.884833 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:34:43.884833 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:34:43.884833 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 17:34:43.884833 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 17:34:43.884833 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 17:34:43.885905 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 17:34:43.885905 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:34:43.885905 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:34:43.885905 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:34:43.885905 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:34:43.885905 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:34:43.885905 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:34:43.885905 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:34:43.885905 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:34:43.885905 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:34:43.885905 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:34:43.885905 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:34:43.885905 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:34:43.885905 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:34:43.885905 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:34:43.885905 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:34:43.885905 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:34:43.887015 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 17:34:43.887015 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 17:34:43.887015 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 17:34:43.887015 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:34:43.887015 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 630/1283] 10.222.0.89 () {56 vars in 1108 bytes} [Thu Jul 28 17:34:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:34:43.892675 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f779750c-3c45-4814-bed5-5ab2dbd07450 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:34:43.988269 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f779750c-3c45-4814-bed5-5ab2dbd07450 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:34:44.022308 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef18bb5-acc4-4715-a6d0-42a8e90e55a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:34:44.022906 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef18bb5-acc4-4715-a6d0-42a8e90e55a4 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:34:44.023184 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef18bb5-acc4-4715-a6d0-42a8e90e55a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:34:44.023458 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef18bb5-acc4-4715-a6d0-42a8e90e55a4 None admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:34:44.024135 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef18bb5-acc4-4715-a6d0-42a8e90e55a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:34:44.024439 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef18bb5-acc4-4715-a6d0-42a8e90e55a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:34:44.025826 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef18bb5-acc4-4715-a6d0-42a8e90e55a4 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:34:44.036736 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 654/1284] 10.222.0.89 () {56 vars in 1109 bytes} [Thu Jul 28 17:34:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:34:44.042464 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071ade10-e11c-405c-900d-b5095531df46 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:34:44.141057 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071ade10-e11c-405c-900d-b5095531df46 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:34:44.155120 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:34:44.175662 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6db5251-e805-4b7e-a1bf-a98603407754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:34:44.176300 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6db5251-e805-4b7e-a1bf-a98603407754 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:34:44.176577 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6db5251-e805-4b7e-a1bf-a98603407754 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:34:44.176847 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6db5251-e805-4b7e-a1bf-a98603407754 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:34:44.183425 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6db5251-e805-4b7e-a1bf-a98603407754 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:34:44.184879 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6db5251-e805-4b7e-a1bf-a98603407754 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:34:44.196815 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application [None req-a6db5251-e805-4b7e-a1bf-a98603407754 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:34:44.196815 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:34:44.196815 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:34:44.196815 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:34:44.196815 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:34:44.196815 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:34:44.196815 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:34:44.196815 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:34:44.196815 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:34:44.196815 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:34:44.196815 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:34:44.196815 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:34:44.196815 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 17:34:44.196815 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 17:34:44.196815 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 17:34:44.197972 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 17:34:44.197972 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:34:44.197972 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:34:44.197972 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:34:44.197972 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:34:44.197972 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:34:44.197972 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:34:44.197972 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:34:44.197972 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application with Lock( Jul 28 17:34:44.197972 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:34:44.197972 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:34:44.197972 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:34:44.197972 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:34:44.197972 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:34:44.197972 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:34:44.197972 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:34:44.197972 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:34:44.199095 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 17:34:44.199095 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 17:34:44.199095 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 17:34:44.199095 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 17:34:44.199095 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 17:34:44.199095 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 17:34:44.199095 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 17:34:44.199095 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: ERROR keystone.server.flask.application Jul 28 17:34:44.199095 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 631/1285] 10.222.0.89 () {56 vars in 1111 bytes} [Thu Jul 28 17:34:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:34:44.204159 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef18bb5-acc4-4715-a6d0-42a8e90e55a4 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:34:44.252960 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:34:44.302309 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef18bb5-acc4-4715-a6d0-42a8e90e55a4 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:34:44.334892 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f9d731-fd12-44b0-a710-0758284f5ec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:34:44.335491 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f9d731-fd12-44b0-a710-0758284f5ec1 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:34:44.335770 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f9d731-fd12-44b0-a710-0758284f5ec1 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:34:44.336041 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f9d731-fd12-44b0-a710-0758284f5ec1 None admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:34:44.336771 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f9d731-fd12-44b0-a710-0758284f5ec1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:34:44.337074 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f9d731-fd12-44b0-a710-0758284f5ec1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:34:44.338512 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f9d731-fd12-44b0-a710-0758284f5ec1 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:34:44.350069 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 655/1286] 10.222.0.89 () {56 vars in 1148 bytes} [Thu Jul 28 17:34:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:34:44.356322 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6db5251-e805-4b7e-a1bf-a98603407754 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:34:44.455424 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6db5251-e805-4b7e-a1bf-a98603407754 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:34:44.489176 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55d4b47-1c47-40a9-a36b-35980eb2d68e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:34:44.489808 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55d4b47-1c47-40a9-a36b-35980eb2d68e None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:34:44.490087 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55d4b47-1c47-40a9-a36b-35980eb2d68e None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:34:44.490355 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55d4b47-1c47-40a9-a36b-35980eb2d68e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:34:44.491188 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55d4b47-1c47-40a9-a36b-35980eb2d68e None admin] RBAC: Adding query filter params (role.id=73e457231a824ea98c28d22610be640c, scope.project.id=04434e8535a64044996316f1d17dc0bc, user.id=e4bd290018394b0d955cd376ce35b77f) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:34:44.491493 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55d4b47-1c47-40a9-a36b-35980eb2d68e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:34:44.492912 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55d4b47-1c47-40a9-a36b-35980eb2d68e None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:34:44.498589 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 632/1287] 10.222.0.89 () {56 vars in 1374 bytes} [Thu Jul 28 17:34:44 2022] GET /identity/v3/role_assignments?user.id=e4bd290018394b0d955cd376ce35b77f&scope.project.id=04434e8535a64044996316f1d17dc0bc&role.id=73e457231a824ea98c28d22610be640c => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:34:46.553620 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955c87a6-97ab-491a-92a9-76f291113b2c None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:34:46.554194 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955c87a6-97ab-491a-92a9-76f291113b2c None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:34:46.554194 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955c87a6-97ab-491a-92a9-76f291113b2c None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:34:46.555533 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 656/1288] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:34:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:34:46.561044 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44782db-cdf3-432f-9de4-f114e7afc0af None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:34:46.562190 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44782db-cdf3-432f-9de4-f114e7afc0af None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:34:46.562190 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44782db-cdf3-432f-9de4-f114e7afc0af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:34:46.582664 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-e44782db-cdf3-432f-9de4-f114e7afc0af None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:34:46.583991 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-e44782db-cdf3-432f-9de4-f114e7afc0af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:34:46.615495 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:34:46.655435 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 633/1289] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:34:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:34:46.677975 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955c87a6-97ab-491a-92a9-76f291113b2c None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:34:46.784573 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955c87a6-97ab-491a-92a9-76f291113b2c None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:34:46.795411 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:34:46.820499 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a09977-f18b-46da-a92e-42b7952b6acd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:34:46.821067 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a09977-f18b-46da-a92e-42b7952b6acd None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:34:46.821375 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a09977-f18b-46da-a92e-42b7952b6acd None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:34:46.821651 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a09977-f18b-46da-a92e-42b7952b6acd None admin] PATH_INFO: `/v3/services/placement` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:34:46.822284 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a09977-f18b-46da-a92e-42b7952b6acd None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:34:46.823662 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a09977-f18b-46da-a92e-42b7952b6acd None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:34:46.840273 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-95a09977-f18b-46da-a92e-42b7952b6acd None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 17:34:46.840273 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:34:46.840273 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 17:34:46.840273 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 17:34:46.840273 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 17:34:46.840273 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 17:34:46.840273 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 17:34:46.840273 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:34:46.840273 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 17:34:46.840273 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:34:46.840273 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 17:34:46.840273 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:34:46.840273 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:34:46.840273 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:34:46.840273 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:34:46.840273 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:34:46.840273 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:34:46.841577 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:34:46.841577 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:34:46.841577 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:34:46.841577 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:34:46.841577 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:34:46.841577 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:34:46.841577 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 17:34:46.841577 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 17:34:46.841577 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 17:34:46.841577 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 17:34:46.841577 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:34:46.841577 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:34:46.841577 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:34:46.841577 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:34:46.841577 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:34:46.841577 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:34:46.842481 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:34:46.842481 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( Jul 28 17:34:46.842481 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:34:46.842481 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:34:46.842481 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:34:46.842481 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:34:46.842481 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:34:46.842481 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:34:46.842481 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:34:46.842481 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:34:46.842481 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 17:34:46.842481 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:34:46.842481 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 17:34:46.842481 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:34:46.842481 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 657/1290] 10.222.0.89 () {56 vars in 1117 bytes} [Thu Jul 28 17:34:46 2022] GET /identity/v3/services/placement => generated 90 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:34:46.848377 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44782db-cdf3-432f-9de4-f114e7afc0af None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:34:46.954140 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44782db-cdf3-432f-9de4-f114e7afc0af None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:34:46.988454 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8072b45-14ef-4e6f-8ff5-617cd702a137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:34:46.989088 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8072b45-14ef-4e6f-8ff5-617cd702a137 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:34:46.989400 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8072b45-14ef-4e6f-8ff5-617cd702a137 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:34:46.989676 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8072b45-14ef-4e6f-8ff5-617cd702a137 None admin] PATH_INFO: `/v3/services` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:34:46.990360 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8072b45-14ef-4e6f-8ff5-617cd702a137 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:34:46.990665 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8072b45-14ef-4e6f-8ff5-617cd702a137 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:34:46.992032 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8072b45-14ef-4e6f-8ff5-617cd702a137 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:34:46.996895 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 634/1291] 10.222.0.89 () {56 vars in 1116 bytes} [Thu Jul 28 17:34:46 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:34:47.004136 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a09977-f18b-46da-a92e-42b7952b6acd None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:34:47.138184 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a09977-f18b-46da-a92e-42b7952b6acd None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:34:47.172049 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be3c1a7-5e0f-40f5-a6b5-be0a64426954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:34:47.172664 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be3c1a7-5e0f-40f5-a6b5-be0a64426954 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:34:47.172945 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be3c1a7-5e0f-40f5-a6b5-be0a64426954 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:34:47.173263 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be3c1a7-5e0f-40f5-a6b5-be0a64426954 None admin] PATH_INFO: `/v3/services` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:34:47.173934 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be3c1a7-5e0f-40f5-a6b5-be0a64426954 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:34:47.174239 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be3c1a7-5e0f-40f5-a6b5-be0a64426954 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:34:47.175616 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be3c1a7-5e0f-40f5-a6b5-be0a64426954 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:34:47.180245 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 658/1292] 10.222.0.89 () {56 vars in 1116 bytes} [Thu Jul 28 17:34:47 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:34:49.164219 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e542cce-6f56-466e-99dc-59af8173151f None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:34:49.164737 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e542cce-6f56-466e-99dc-59af8173151f None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:34:49.164737 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e542cce-6f56-466e-99dc-59af8173151f None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:34:49.166081 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 635/1293] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:34:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:34:49.171412 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6cf0db-8285-4695-a660-45c61bbc410d None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:34:49.171817 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6cf0db-8285-4695-a660-45c61bbc410d None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:34:49.172004 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6cf0db-8285-4695-a660-45c61bbc410d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:34:49.191417 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-2d6cf0db-8285-4695-a660-45c61bbc410d None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:34:49.192637 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-2d6cf0db-8285-4695-a660-45c61bbc410d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:34:49.222380 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:34:49.260189 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 659/1294] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:34:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:34:49.283272 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e542cce-6f56-466e-99dc-59af8173151f None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:34:49.314917 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:34:49.383960 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e542cce-6f56-466e-99dc-59af8173151f None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:34:49.416674 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af160162-dd0d-403d-b322-f233ec4ae08f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:34:49.417274 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af160162-dd0d-403d-b322-f233ec4ae08f None admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:34:49.417557 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af160162-dd0d-403d-b322-f233ec4ae08f None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:34:49.417834 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af160162-dd0d-403d-b322-f233ec4ae08f None admin] PATH_INFO: `/v3/services` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:34:49.418563 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af160162-dd0d-403d-b322-f233ec4ae08f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:34:49.419920 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af160162-dd0d-403d-b322-f233ec4ae08f None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:34:49.435478 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 636/1295] 10.222.0.89 () {60 vars in 1141 bytes} [Thu Jul 28 17:34:49 2022] POST /identity/v3/services => generated 246 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:34:51.472374 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54cda7a-f65f-4e3d-a766-294894fbbf02 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:34:51.472914 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54cda7a-f65f-4e3d-a766-294894fbbf02 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:34:51.472914 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54cda7a-f65f-4e3d-a766-294894fbbf02 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:34:51.474309 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 660/1296] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:34:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:34:51.479272 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7fd055-943e-4b3a-ad6a-018dc3c9f962 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:34:51.479672 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7fd055-943e-4b3a-ad6a-018dc3c9f962 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:34:51.479829 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7fd055-943e-4b3a-ad6a-018dc3c9f962 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:34:51.498311 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-2f7fd055-943e-4b3a-ad6a-018dc3c9f962 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:34:51.500289 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-2f7fd055-943e-4b3a-ad6a-018dc3c9f962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:34:51.578966 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 637/1297] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:34:51 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 17:34:51.602249 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54cda7a-f65f-4e3d-a766-294894fbbf02 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:34:51.701468 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54cda7a-f65f-4e3d-a766-294894fbbf02 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:34:51.736648 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e45a852-719d-48a1-9fd2-f7c7ff5b6c7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:34:51.737231 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e45a852-719d-48a1-9fd2-f7c7ff5b6c7a None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:34:51.737513 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e45a852-719d-48a1-9fd2-f7c7ff5b6c7a None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:34:51.737787 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e45a852-719d-48a1-9fd2-f7c7ff5b6c7a None admin] PATH_INFO: `/v3/services/placement` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:34:51.738408 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e45a852-719d-48a1-9fd2-f7c7ff5b6c7a None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:34:51.739772 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e45a852-719d-48a1-9fd2-f7c7ff5b6c7a None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:34:51.748812 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-6e45a852-719d-48a1-9fd2-f7c7ff5b6c7a None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 17:34:51.748812 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:34:51.748812 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 17:34:51.748812 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 17:34:51.748812 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 17:34:51.748812 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 17:34:51.748812 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 17:34:51.748812 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:34:51.748812 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 17:34:51.748812 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:34:51.748812 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 17:34:51.748812 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:34:51.748812 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:34:51.748812 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:34:51.748812 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:34:51.748812 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:34:51.748812 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:34:51.750629 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:34:51.750629 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:34:51.750629 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:34:51.750629 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:34:51.750629 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:34:51.750629 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:34:51.750629 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 17:34:51.750629 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 17:34:51.750629 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 17:34:51.750629 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 17:34:51.750629 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:34:51.750629 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:34:51.750629 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:34:51.750629 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:34:51.750629 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:34:51.750629 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:34:51.752111 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:34:51.752111 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( Jul 28 17:34:51.752111 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:34:51.752111 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:34:51.752111 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:34:51.752111 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:34:51.752111 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:34:51.752111 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:34:51.752111 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:34:51.752111 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:34:51.752111 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 17:34:51.752111 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:34:51.752111 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 17:34:51.752111 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:34:51.752111 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 661/1298] 10.222.0.89 () {56 vars in 1117 bytes} [Thu Jul 28 17:34:51 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:34:51.757233 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7fd055-943e-4b3a-ad6a-018dc3c9f962 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:34:51.774869 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:34:51.861640 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7fd055-943e-4b3a-ad6a-018dc3c9f962 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:34:51.894609 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a420a719-75cb-4525-a8fe-d68d81bf70d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:34:51.895218 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a420a719-75cb-4525-a8fe-d68d81bf70d5 None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:34:51.895494 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a420a719-75cb-4525-a8fe-d68d81bf70d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:34:51.895766 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a420a719-75cb-4525-a8fe-d68d81bf70d5 None admin] PATH_INFO: `/v3/services` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:34:51.896440 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a420a719-75cb-4525-a8fe-d68d81bf70d5 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:34:51.896742 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a420a719-75cb-4525-a8fe-d68d81bf70d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:34:51.898130 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a420a719-75cb-4525-a8fe-d68d81bf70d5 None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:34:51.903156 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 638/1299] 10.222.0.89 () {56 vars in 1116 bytes} [Thu Jul 28 17:34:51 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:34:51.909706 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e45a852-719d-48a1-9fd2-f7c7ff5b6c7a None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:34:51.941257 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:34:52.010944 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e45a852-719d-48a1-9fd2-f7c7ff5b6c7a None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:34:52.044240 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb3a10d-e7eb-44ab-8c6a-a3e48448d5d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:34:52.044835 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb3a10d-e7eb-44ab-8c6a-a3e48448d5d0 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:34:52.045111 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb3a10d-e7eb-44ab-8c6a-a3e48448d5d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:34:52.045408 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb3a10d-e7eb-44ab-8c6a-a3e48448d5d0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:34:52.046151 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb3a10d-e7eb-44ab-8c6a-a3e48448d5d0 None admin] RBAC: Adding query filter params (interface=public, service_id=6e517a29de7949f4b4e0261098a440b0, region_id=RegionOne) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:34:52.046482 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb3a10d-e7eb-44ab-8c6a-a3e48448d5d0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:34:52.048719 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb3a10d-e7eb-44ab-8c6a-a3e48448d5d0 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:34:52.053609 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 662/1300] 10.222.0.89 () {56 vars in 1251 bytes} [Thu Jul 28 17:34:51 2022] GET /identity/v3/endpoints?service_id=6e517a29de7949f4b4e0261098a440b0&interface=public®ion_id=RegionOne => generated 194 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:34:52.059383 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a420a719-75cb-4525-a8fe-d68d81bf70d5 None admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:34:52.140568 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:34:52.160781 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a420a719-75cb-4525-a8fe-d68d81bf70d5 None admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:34:52.198664 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b028e954-107f-40ca-88a0-ad377b49794a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:34:52.199512 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b028e954-107f-40ca-88a0-ad377b49794a None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:34:52.199794 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b028e954-107f-40ca-88a0-ad377b49794a None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:34:52.200069 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b028e954-107f-40ca-88a0-ad377b49794a None admin] PATH_INFO: `/v3/services` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:34:52.200729 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b028e954-107f-40ca-88a0-ad377b49794a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:34:52.202126 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b028e954-107f-40ca-88a0-ad377b49794a None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:34:52.208802 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 639/1301] 10.222.0.89 () {56 vars in 1087 bytes} [Thu Jul 28 17:34:52 2022] GET /identity/v3/services => generated 2191 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:34:54.196393 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e51c06-232b-4432-89a7-8aeefc33d1ae None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:34:54.196934 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e51c06-232b-4432-89a7-8aeefc33d1ae None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:34:54.196934 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e51c06-232b-4432-89a7-8aeefc33d1ae None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:34:54.198291 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 663/1302] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:34:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:34:54.204251 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feff205-ef45-4e22-a580-ab884b8b8adb None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:34:54.204673 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feff205-ef45-4e22-a580-ab884b8b8adb None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:34:54.204782 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feff205-ef45-4e22-a580-ab884b8b8adb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:34:54.222585 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-3feff205-ef45-4e22-a580-ab884b8b8adb None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:34:54.223772 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-3feff205-ef45-4e22-a580-ab884b8b8adb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:34:54.288916 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 640/1303] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:34:54 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 17:34:54.311219 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e51c06-232b-4432-89a7-8aeefc33d1ae None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:34:54.405647 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:34:54.413258 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e51c06-232b-4432-89a7-8aeefc33d1ae None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:34:54.445971 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca297dd9-ef1c-48df-9951-ed9bbf4aef74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:34:54.446646 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca297dd9-ef1c-48df-9951-ed9bbf4aef74 None admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:34:54.446868 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca297dd9-ef1c-48df-9951-ed9bbf4aef74 None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:34:54.447136 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca297dd9-ef1c-48df-9951-ed9bbf4aef74 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:34:54.447748 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca297dd9-ef1c-48df-9951-ed9bbf4aef74 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:34:54.449104 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca297dd9-ef1c-48df-9951-ed9bbf4aef74 None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:34:54.458304 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application [None req-ca297dd9-ef1c-48df-9951-ed9bbf4aef74 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 17:34:54.458304 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:34:54.458304 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 17:34:54.458304 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 17:34:54.458304 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 17:34:54.458304 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 17:34:54.458304 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 17:34:54.458304 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:34:54.458304 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 17:34:54.458304 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:34:54.458304 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 17:34:54.458304 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:34:54.458304 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 17:34:54.458304 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 17:34:54.458304 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 17:34:54.458304 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 17:34:54.458304 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 17:34:54.459651 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 17:34:54.459651 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 17:34:54.459651 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 17:34:54.459651 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 17:34:54.459651 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 17:34:54.459651 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 17:34:54.459651 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 17:34:54.459651 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 17:34:54.459651 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 17:34:54.459651 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 17:34:54.459651 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 17:34:54.459651 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 17:34:54.459651 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 17:34:54.459651 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 17:34:54.459651 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 17:34:54.459651 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 17:34:54.460901 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 17:34:54.460901 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application with Lock( Jul 28 17:34:54.460901 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 17:34:54.460901 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self._enter() Jul 28 17:34:54.460901 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 17:34:54.460901 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 17:34:54.460901 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 17:34:54.460901 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application return self.creator() Jul 28 17:34:54.460901 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 17:34:54.460901 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application created_value = creator( Jul 28 17:34:54.460901 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 17:34:54.460901 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 17:34:54.460901 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 17:34:54.460901 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: ERROR keystone.server.flask.application Jul 28 17:34:54.460901 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 664/1304] 10.222.0.89 () {56 vars in 1117 bytes} [Thu Jul 28 17:34:54 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 17:34:54.465610 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3feff205-ef45-4e22-a580-ab884b8b8adb None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:34:54.545772 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:34:54.570234 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3feff205-ef45-4e22-a580-ab884b8b8adb None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:34:54.608506 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a9de49-1938-4864-925c-2546d1cd9d9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:34:54.609652 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a9de49-1938-4864-925c-2546d1cd9d9b None admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:34:54.609936 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a9de49-1938-4864-925c-2546d1cd9d9b None admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:34:54.610216 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a9de49-1938-4864-925c-2546d1cd9d9b None admin] PATH_INFO: `/v3/services` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:34:54.610897 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a9de49-1938-4864-925c-2546d1cd9d9b None admin] RBAC: Adding query filter params (name=placement) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:34:54.611198 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a9de49-1938-4864-925c-2546d1cd9d9b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:34:54.612566 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a9de49-1938-4864-925c-2546d1cd9d9b None admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:34:54.617644 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 641/1305] 10.222.0.89 () {56 vars in 1116 bytes} [Thu Jul 28 17:34:54 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:34:54.624357 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca297dd9-ef1c-48df-9951-ed9bbf4aef74 None admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:34:54.727243 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca297dd9-ef1c-48df-9951-ed9bbf4aef74 None admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:34:54.761203 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4bd54c-a806-4451-bf61-35a1f8613fce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:34:54.761815 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4bd54c-a806-4451-bf61-35a1f8613fce None admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:34:54.762095 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4bd54c-a806-4451-bf61-35a1f8613fce None admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:34:54.762367 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4bd54c-a806-4451-bf61-35a1f8613fce None admin] PATH_INFO: `/v3/endpoints` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:34:54.763068 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4bd54c-a806-4451-bf61-35a1f8613fce None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:34:54.764419 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4bd54c-a806-4451-bf61-35a1f8613fce None admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:34:54.783834 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 665/1306] 10.222.0.89 () {60 vars in 1144 bytes} [Thu Jul 28 17:34:54 2022] POST /identity/v3/endpoints => generated 331 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:35:04.614764 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=30 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:35:04.779411 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:35:58.896116 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a667f0-9e9d-48af-95fe-6343e8dd7c9c None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:35:58.896600 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a667f0-9e9d-48af-95fe-6343e8dd7c9c None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:35:58.896696 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a667f0-9e9d-48af-95fe-6343e8dd7c9c None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:35:58.900555 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 642/1307] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:35:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:35:58.906962 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c36cfa5-9d4e-4f72-88b5-d04222d2dd50 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:35:58.908492 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c36cfa5-9d4e-4f72-88b5-d04222d2dd50 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:35:58.909093 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c36cfa5-9d4e-4f72-88b5-d04222d2dd50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:35:59.056246 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-5c36cfa5-9d4e-4f72-88b5-d04222d2dd50 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:35:59.057503 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-5c36cfa5-9d4e-4f72-88b5-d04222d2dd50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:35:59.176426 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 666/1308] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:35:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 17:35:59.227438 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c2c6f1-9a4a-4feb-835f-86b17a126b04 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:35:59.227798 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c2c6f1-9a4a-4feb-835f-86b17a126b04 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:35:59.228182 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c2c6f1-9a4a-4feb-835f-86b17a126b04 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:35:59.229704 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 643/1309] 10.222.0.89 () {54 vars in 965 bytes} [Thu Jul 28 17:35:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:35:59.236657 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2febbd31-dcec-4d4c-a618-bf047a9d8a15 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:35:59.236989 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2febbd31-dcec-4d4c-a618-bf047a9d8a15 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:35:59.237299 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2febbd31-dcec-4d4c-a618-bf047a9d8a15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:35:59.331343 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-2febbd31-dcec-4d4c-a618-bf047a9d8a15 None None] MFA Rules not processed for user `76817cd541294331b0023069092e8e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:35:59.332790 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-2febbd31-dcec-4d4c-a618-bf047a9d8a15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:35:59.359642 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:35:59.427101 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 667/1310] 10.222.0.89 () {58 vars in 1062 bytes} [Thu Jul 28 17:35:59 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:35:59.436173 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c2c6f1-9a4a-4feb-835f-86b17a126b04 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:35:59.438175 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-36c2c6f1-9a4a-4feb-835f-86b17a126b04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:35:59.566178 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c2c6f1-9a4a-4feb-835f-86b17a126b04 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:35:59.651947 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586dabb0-f679-45e9-9e05-b3005b22ebf0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76817cd541294331b0023069092e8e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:35:59.652914 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586dabb0-f679-45e9-9e05-b3005b22ebf0 service neutron] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:35:59.653215 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586dabb0-f679-45e9-9e05-b3005b22ebf0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:35:59.653492 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586dabb0-f679-45e9-9e05-b3005b22ebf0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:35:59.655280 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-586dabb0-f679-45e9-9e05-b3005b22ebf0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:35:59.718707 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:35:59.787987 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586dabb0-f679-45e9-9e05-b3005b22ebf0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:35:59.789476 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586dabb0-f679-45e9-9e05-b3005b22ebf0 service neutron] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:35:59.816906 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 644/1311] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:35:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 17:36:02.169943 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b93267-1d43-4101-b23d-67ba8b843ecf None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:36:02.170477 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b93267-1d43-4101-b23d-67ba8b843ecf None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:36:02.170477 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b93267-1d43-4101-b23d-67ba8b843ecf None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:36:02.172009 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 668/1312] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:36:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:36:02.178683 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7856ca-86f9-42d6-901b-93a0cfea80f2 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:36:02.179616 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7856ca-86f9-42d6-901b-93a0cfea80f2 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:36:02.179970 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7856ca-86f9-42d6-901b-93a0cfea80f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:36:02.274969 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-0b7856ca-86f9-42d6-901b-93a0cfea80f2 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:36:02.276212 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-0b7856ca-86f9-42d6-901b-93a0cfea80f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:36:02.390204 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 645/1313] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:36:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:36:02.429807 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b93267-1d43-4101-b23d-67ba8b843ecf None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:36:02.431349 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-47b93267-1d43-4101-b23d-67ba8b843ecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:36:02.550099 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b93267-1d43-4101-b23d-67ba8b843ecf None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:36:02.626966 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:36:02.630254 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b8a84c-920f-4b24-acc8-73d27addb397 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76817cd541294331b0023069092e8e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:36:02.631096 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b8a84c-920f-4b24-acc8-73d27addb397 service neutron] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:36:02.631641 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b8a84c-920f-4b24-acc8-73d27addb397 service neutron] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:36:02.632105 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b8a84c-920f-4b24-acc8-73d27addb397 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:36:02.633822 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-71b8a84c-920f-4b24-acc8-73d27addb397 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:36:02.747748 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b8a84c-920f-4b24-acc8-73d27addb397 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:36:02.749377 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b8a84c-920f-4b24-acc8-73d27addb397 service neutron] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:36:02.769642 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 669/1314] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:36:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:36:05.155692 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05517695-a81f-4280-b84b-ff90a1f78378 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:36:05.156215 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05517695-a81f-4280-b84b-ff90a1f78378 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:36:05.156215 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05517695-a81f-4280-b84b-ff90a1f78378 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:36:05.157577 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 646/1315] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:36:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:36:05.163470 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d37e43-bfcb-480a-996f-33dcd8d1e6a4 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:36:05.163871 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d37e43-bfcb-480a-996f-33dcd8d1e6a4 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:36:05.164028 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d37e43-bfcb-480a-996f-33dcd8d1e6a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:36:05.230079 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-29d37e43-bfcb-480a-996f-33dcd8d1e6a4 None None] MFA Rules not processed for user `7eafa4211ada43a0b09036090ff902f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:36:05.231391 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-29d37e43-bfcb-480a-996f-33dcd8d1e6a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:36:05.282311 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:36:05.327951 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 670/1316] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:36:05 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:36:05.368645 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05517695-a81f-4280-b84b-ff90a1f78378 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:36:05.370336 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-05517695-a81f-4280-b84b-ff90a1f78378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:36:05.382690 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:36:05.493984 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05517695-a81f-4280-b84b-ff90a1f78378 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:36:05.567132 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d20d9a-793a-47c0-ac08-5fd74f15de08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76817cd541294331b0023069092e8e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:36:05.567753 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d20d9a-793a-47c0-ac08-5fd74f15de08 service neutron] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:36:05.568034 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d20d9a-793a-47c0-ac08-5fd74f15de08 service neutron] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:36:05.568303 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d20d9a-793a-47c0-ac08-5fd74f15de08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:36:05.569976 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-93d20d9a-793a-47c0-ac08-5fd74f15de08 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:36:05.630626 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:36:05.678136 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d20d9a-793a-47c0-ac08-5fd74f15de08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:36:05.679578 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d20d9a-793a-47c0-ac08-5fd74f15de08 service neutron] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:36:05.697916 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 647/1317] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:36:05 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:36:08.311709 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e454e3-459c-4aee-8d1d-098bcba5b785 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:36:08.312245 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e454e3-459c-4aee-8d1d-098bcba5b785 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:36:08.312245 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e454e3-459c-4aee-8d1d-098bcba5b785 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:36:08.313648 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 671/1318] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:36:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:36:08.319732 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f221b327-2583-4156-ae55-ff434edda502 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:36:08.320173 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f221b327-2583-4156-ae55-ff434edda502 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:36:08.320603 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f221b327-2583-4156-ae55-ff434edda502 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:36:08.386821 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-f221b327-2583-4156-ae55-ff434edda502 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:36:08.388080 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-f221b327-2583-4156-ae55-ff434edda502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:36:08.484227 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 648/1319] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:36:08 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:36:08.523970 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e454e3-459c-4aee-8d1d-098bcba5b785 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:36:08.525624 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-94e454e3-459c-4aee-8d1d-098bcba5b785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:36:08.631506 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e454e3-459c-4aee-8d1d-098bcba5b785 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:36:08.704952 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5f69ad-ecb1-4a88-bddd-048dfbb4d06f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76817cd541294331b0023069092e8e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:36:08.705596 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5f69ad-ecb1-4a88-bddd-048dfbb4d06f service neutron] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:36:08.705877 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5f69ad-ecb1-4a88-bddd-048dfbb4d06f service neutron] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:36:08.706148 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5f69ad-ecb1-4a88-bddd-048dfbb4d06f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:36:08.707776 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-5e5f69ad-ecb1-4a88-bddd-048dfbb4d06f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:36:08.715146 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:36:08.818036 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5f69ad-ecb1-4a88-bddd-048dfbb4d06f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:36:08.819501 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5f69ad-ecb1-4a88-bddd-048dfbb4d06f service neutron] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:36:08.838004 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 672/1320] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:36:08 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:36:11.565096 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc904a84-f03f-4259-b02c-6c0b3fdec55c None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:36:11.565704 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc904a84-f03f-4259-b02c-6c0b3fdec55c None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:36:11.565704 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc904a84-f03f-4259-b02c-6c0b3fdec55c None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:36:11.567168 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 649/1321] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:36:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:36:11.572770 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e182b76-ea00-4b9e-a440-6eacda2d08a1 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:36:11.573242 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e182b76-ea00-4b9e-a440-6eacda2d08a1 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:36:11.573509 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e182b76-ea00-4b9e-a440-6eacda2d08a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:36:11.646045 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-0e182b76-ea00-4b9e-a440-6eacda2d08a1 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:36:11.647285 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-0e182b76-ea00-4b9e-a440-6eacda2d08a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:36:11.708390 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:36:11.754058 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 673/1322] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:36:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:36:11.809795 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5e57b1-a988-477d-b12d-f77b823a7896 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:36:11.810056 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5e57b1-a988-477d-b12d-f77b823a7896 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:36:11.810383 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5e57b1-a988-477d-b12d-f77b823a7896 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:36:11.811804 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 650/1323] 10.222.0.89 () {54 vars in 965 bytes} [Thu Jul 28 17:36:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:36:11.820441 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaa4fa1-7ba8-4b5a-b255-abbb0927a581 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:36:11.820663 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaa4fa1-7ba8-4b5a-b255-abbb0927a581 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:36:11.820903 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaa4fa1-7ba8-4b5a-b255-abbb0927a581 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:36:11.921474 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-cbaa4fa1-7ba8-4b5a-b255-abbb0927a581 None None] MFA Rules not processed for user `76817cd541294331b0023069092e8e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:36:11.923431 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-cbaa4fa1-7ba8-4b5a-b255-abbb0927a581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:36:12.038904 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 674/1324] 10.222.0.89 () {58 vars in 1062 bytes} [Thu Jul 28 17:36:11 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 17:36:12.048266 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5e57b1-a988-477d-b12d-f77b823a7896 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:36:12.050223 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-0c5e57b1-a988-477d-b12d-f77b823a7896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:36:12.070332 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:36:12.162138 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5e57b1-a988-477d-b12d-f77b823a7896 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:36:12.240063 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9733b66a-4106-4273-a6e8-8724a7ecdf61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76817cd541294331b0023069092e8e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:36:12.240678 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9733b66a-4106-4273-a6e8-8724a7ecdf61 service neutron] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:36:12.240955 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9733b66a-4106-4273-a6e8-8724a7ecdf61 service neutron] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:36:12.241259 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9733b66a-4106-4273-a6e8-8724a7ecdf61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:36:12.242881 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-9733b66a-4106-4273-a6e8-8724a7ecdf61 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:36:12.317453 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:36:12.355509 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9733b66a-4106-4273-a6e8-8724a7ecdf61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:36:12.356949 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9733b66a-4106-4273-a6e8-8724a7ecdf61 service neutron] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:36:12.375341 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 651/1325] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:36:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:36:14.719285 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a25ccef-d5c4-408f-b657-0d404b233ea2 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:36:14.719893 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a25ccef-d5c4-408f-b657-0d404b233ea2 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:36:14.719893 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a25ccef-d5c4-408f-b657-0d404b233ea2 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:36:14.721984 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 675/1326] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:36:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:36:14.727307 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d841559-850b-4a49-b34b-36025818a750 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:36:14.727768 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d841559-850b-4a49-b34b-36025818a750 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:36:14.727870 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d841559-850b-4a49-b34b-36025818a750 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:36:14.795485 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-4d841559-850b-4a49-b34b-36025818a750 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:36:14.796618 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-4d841559-850b-4a49-b34b-36025818a750 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:36:14.908346 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 652/1327] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:36:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:36:14.946025 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a25ccef-d5c4-408f-b657-0d404b233ea2 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:36:14.947573 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-2a25ccef-d5c4-408f-b657-0d404b233ea2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:36:15.009971 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:36:15.054748 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a25ccef-d5c4-408f-b657-0d404b233ea2 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:36:15.130580 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b5f2cb-d2a9-42a1-b579-e515434b1823 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76817cd541294331b0023069092e8e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:36:15.131196 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b5f2cb-d2a9-42a1-b579-e515434b1823 service neutron] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:36:15.131471 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b5f2cb-d2a9-42a1-b579-e515434b1823 service neutron] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:36:15.131740 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b5f2cb-d2a9-42a1-b579-e515434b1823 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:36:15.133390 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-e4b5f2cb-d2a9-42a1-b579-e515434b1823 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:36:15.243117 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:36:15.245526 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b5f2cb-d2a9-42a1-b579-e515434b1823 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:36:15.247590 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b5f2cb-d2a9-42a1-b579-e515434b1823 service neutron] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:36:15.266293 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 676/1328] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:36:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:36:17.342602 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c60acb-6435-4f9a-a6e7-2c85c9403faa None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:36:17.343125 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c60acb-6435-4f9a-a6e7-2c85c9403faa None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:36:17.343125 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c60acb-6435-4f9a-a6e7-2c85c9403faa None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:36:17.344474 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 653/1329] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:36:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:36:17.351619 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3f3a38-e854-42f5-93f2-865b15dd164f None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:36:17.352162 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3f3a38-e854-42f5-93f2-865b15dd164f None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:36:17.352162 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3f3a38-e854-42f5-93f2-865b15dd164f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:36:17.422660 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-4f3f3a38-e854-42f5-93f2-865b15dd164f None None] MFA Rules not processed for user `7eafa4211ada43a0b09036090ff902f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:36:17.424609 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-4f3f3a38-e854-42f5-93f2-865b15dd164f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:36:17.524189 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 677/1330] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:36:17 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 17:36:17.564336 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c60acb-6435-4f9a-a6e7-2c85c9403faa None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:36:17.566237 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-65c60acb-6435-4f9a-a6e7-2c85c9403faa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:36:17.605664 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:36:17.690153 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c60acb-6435-4f9a-a6e7-2c85c9403faa None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:36:17.763027 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d2b5ae-60b8-4f97-94f1-8244fbc6aed6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76817cd541294331b0023069092e8e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:36:17.763769 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d2b5ae-60b8-4f97-94f1-8244fbc6aed6 service neutron] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:36:17.764047 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d2b5ae-60b8-4f97-94f1-8244fbc6aed6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:36:17.764319 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d2b5ae-60b8-4f97-94f1-8244fbc6aed6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:36:17.766000 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-39d2b5ae-60b8-4f97-94f1-8244fbc6aed6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:36:17.847411 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:36:17.881452 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d2b5ae-60b8-4f97-94f1-8244fbc6aed6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:36:17.883337 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d2b5ae-60b8-4f97-94f1-8244fbc6aed6 service neutron] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:36:17.903309 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 654/1331] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:36:17 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:36:20.341441 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0d1c2c-8d0f-4e6e-a472-d81aa81d777a None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:36:20.342017 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0d1c2c-8d0f-4e6e-a472-d81aa81d777a None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:36:20.342017 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0d1c2c-8d0f-4e6e-a472-d81aa81d777a None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:36:20.343337 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 678/1332] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:36:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:36:20.349531 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1232dde0-e1f9-4233-a479-e0eba375f13d None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:36:20.350077 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1232dde0-e1f9-4233-a479-e0eba375f13d None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:36:20.350077 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1232dde0-e1f9-4233-a479-e0eba375f13d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:36:20.418205 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-1232dde0-e1f9-4233-a479-e0eba375f13d None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:36:20.419415 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-1232dde0-e1f9-4233-a479-e0eba375f13d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:36:20.521341 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 655/1333] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:36:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:36:20.553683 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0d1c2c-8d0f-4e6e-a472-d81aa81d777a None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:36:20.555240 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-4e0d1c2c-8d0f-4e6e-a472-d81aa81d777a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:36:20.628767 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:36:20.669106 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0d1c2c-8d0f-4e6e-a472-d81aa81d777a None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:36:20.762917 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d900dc-ee70-4236-b845-3797295a19f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76817cd541294331b0023069092e8e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:36:20.763522 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d900dc-ee70-4236-b845-3797295a19f1 service neutron] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:36:20.763800 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d900dc-ee70-4236-b845-3797295a19f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:36:20.764070 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d900dc-ee70-4236-b845-3797295a19f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:36:20.765848 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-e4d900dc-ee70-4236-b845-3797295a19f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:36:20.874351 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d900dc-ee70-4236-b845-3797295a19f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:36:20.875699 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d900dc-ee70-4236-b845-3797295a19f1 service neutron] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:36:20.886415 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:36:20.895078 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 679/1334] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:36:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:36:23.010918 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304ebfc5-4821-4034-993d-88a8a88516d2 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:36:23.011448 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304ebfc5-4821-4034-993d-88a8a88516d2 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:36:23.011448 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304ebfc5-4821-4034-993d-88a8a88516d2 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:36:23.012819 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 656/1335] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:36:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:36:23.017930 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb04c5e4-cbfb-4b6b-ab1f-db856398238b None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:36:23.018335 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb04c5e4-cbfb-4b6b-ab1f-db856398238b None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:36:23.018483 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb04c5e4-cbfb-4b6b-ab1f-db856398238b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:36:23.086397 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-eb04c5e4-cbfb-4b6b-ab1f-db856398238b None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:36:23.087551 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-eb04c5e4-cbfb-4b6b-ab1f-db856398238b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:36:23.185540 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 680/1336] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:36:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 17:36:23.226213 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304ebfc5-4821-4034-993d-88a8a88516d2 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:36:23.227764 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-304ebfc5-4821-4034-993d-88a8a88516d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:36:23.269013 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:36:23.333672 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304ebfc5-4821-4034-993d-88a8a88516d2 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:36:23.407660 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ae81e4-21fa-478b-949f-4715f33b2534 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76817cd541294331b0023069092e8e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:36:23.408425 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ae81e4-21fa-478b-949f-4715f33b2534 service neutron] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:36:23.408555 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ae81e4-21fa-478b-949f-4715f33b2534 service neutron] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:36:23.408762 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ae81e4-21fa-478b-949f-4715f33b2534 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:36:23.410445 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-10ae81e4-21fa-478b-949f-4715f33b2534 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:36:23.493360 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:36:23.519922 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ae81e4-21fa-478b-949f-4715f33b2534 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:36:23.521382 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ae81e4-21fa-478b-949f-4715f33b2534 service neutron] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:36:23.539578 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 657/1337] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:36:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:36:26.032964 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f418872b-98f9-4841-afef-7fb1c3706818 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:36:26.033539 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f418872b-98f9-4841-afef-7fb1c3706818 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:36:26.033635 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f418872b-98f9-4841-afef-7fb1c3706818 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:36:26.035521 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 681/1338] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:36:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:36:26.040649 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdd374a-111e-475c-bd39-1a189ec8eba9 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:36:26.041102 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdd374a-111e-475c-bd39-1a189ec8eba9 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:36:26.041239 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdd374a-111e-475c-bd39-1a189ec8eba9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:36:26.119467 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-acdd374a-111e-475c-bd39-1a189ec8eba9 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:36:26.120720 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-acdd374a-111e-475c-bd39-1a189ec8eba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:36:26.218289 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 658/1339] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:36:26 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:36:26.255388 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f418872b-98f9-4841-afef-7fb1c3706818 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:36:26.257310 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-f418872b-98f9-4841-afef-7fb1c3706818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:36:26.332555 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:36:26.360494 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f418872b-98f9-4841-afef-7fb1c3706818 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:36:26.434265 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d671dfa-d44c-4c01-8d02-2daaf2c0b1fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76817cd541294331b0023069092e8e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:36:26.434854 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d671dfa-d44c-4c01-8d02-2daaf2c0b1fe service neutron] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:36:26.435128 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d671dfa-d44c-4c01-8d02-2daaf2c0b1fe service neutron] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:36:26.435394 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d671dfa-d44c-4c01-8d02-2daaf2c0b1fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:36:26.436996 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-0d671dfa-d44c-4c01-8d02-2daaf2c0b1fe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:36:26.549901 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d671dfa-d44c-4c01-8d02-2daaf2c0b1fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:36:26.551429 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d671dfa-d44c-4c01-8d02-2daaf2c0b1fe service neutron] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:36:26.569608 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 682/1340] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:36:26 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:36:30.644670 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44725c5-9031-4d87-baa2-4038f5471045 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:36:30.645235 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44725c5-9031-4d87-baa2-4038f5471045 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:36:30.645235 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44725c5-9031-4d87-baa2-4038f5471045 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:36:30.646533 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 659/1341] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:36:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 17:36:30.651750 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04867cc5-fbde-43f9-a64b-595326fe2712 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:36:30.652176 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04867cc5-fbde-43f9-a64b-595326fe2712 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:36:30.652363 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04867cc5-fbde-43f9-a64b-595326fe2712 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:36:30.657980 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:36:30.720458 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-04867cc5-fbde-43f9-a64b-595326fe2712 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:36:30.722599 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-04867cc5-fbde-43f9-a64b-595326fe2712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:36:30.831798 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 683/1342] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:36:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 17:36:30.872767 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44725c5-9031-4d87-baa2-4038f5471045 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:36:30.874365 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-f44725c5-9031-4d87-baa2-4038f5471045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:36:30.925374 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:36:30.984777 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44725c5-9031-4d87-baa2-4038f5471045 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:36:31.061448 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82566714-14d2-4967-af2d-c2462b95fdc0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76817cd541294331b0023069092e8e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:36:31.062067 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82566714-14d2-4967-af2d-c2462b95fdc0 service neutron] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:36:31.062351 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82566714-14d2-4967-af2d-c2462b95fdc0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:36:31.062627 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82566714-14d2-4967-af2d-c2462b95fdc0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:36:31.064268 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-82566714-14d2-4967-af2d-c2462b95fdc0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:36:31.159303 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:36:31.179615 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82566714-14d2-4967-af2d-c2462b95fdc0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:36:31.180978 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82566714-14d2-4967-af2d-c2462b95fdc0 service neutron] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:36:31.199686 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 660/1343] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:36:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:36:33.845008 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c5a6c6-83de-4946-93cc-57c08901db6f None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:36:33.845614 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c5a6c6-83de-4946-93cc-57c08901db6f None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:36:33.845614 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c5a6c6-83de-4946-93cc-57c08901db6f None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:36:33.847515 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 684/1344] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:36:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:36:33.853653 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9915d0d1-f9f8-4ca6-bf8e-9368437db3da None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:36:33.854251 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9915d0d1-f9f8-4ca6-bf8e-9368437db3da None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:36:33.854251 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9915d0d1-f9f8-4ca6-bf8e-9368437db3da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:36:33.929534 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-9915d0d1-f9f8-4ca6-bf8e-9368437db3da None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:36:33.930787 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-9915d0d1-f9f8-4ca6-bf8e-9368437db3da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:36:34.040262 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 661/1345] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:36:33 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:36:34.078947 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c5a6c6-83de-4946-93cc-57c08901db6f None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:36:34.081143 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-83c5a6c6-83de-4946-93cc-57c08901db6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:36:34.170367 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:36:34.188676 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c5a6c6-83de-4946-93cc-57c08901db6f None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:36:34.262857 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc082a44-f7a4-48bf-b72e-578c268a3e41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76817cd541294331b0023069092e8e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:36:34.263399 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc082a44-f7a4-48bf-b72e-578c268a3e41 service neutron] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:36:34.263679 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc082a44-f7a4-48bf-b72e-578c268a3e41 service neutron] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:36:34.263951 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc082a44-f7a4-48bf-b72e-578c268a3e41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:36:34.265617 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-bc082a44-f7a4-48bf-b72e-578c268a3e41 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:36:34.379655 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc082a44-f7a4-48bf-b72e-578c268a3e41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:36:34.381279 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc082a44-f7a4-48bf-b72e-578c268a3e41 service neutron] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:36:34.400404 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 685/1346] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:36:34 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:36:38.444329 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e00e3c8-2af9-435b-addc-7bf5421647d5 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:36:38.444908 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e00e3c8-2af9-435b-addc-7bf5421647d5 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:36:38.444908 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e00e3c8-2af9-435b-addc-7bf5421647d5 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:36:38.446323 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 662/1347] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:36:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:36:38.451735 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db8c898-aded-47e7-b214-94a3520b89cc None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:36:38.452165 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db8c898-aded-47e7-b214-94a3520b89cc None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:36:38.452281 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db8c898-aded-47e7-b214-94a3520b89cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:36:38.467978 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:36:38.522396 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-1db8c898-aded-47e7-b214-94a3520b89cc None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:36:38.524344 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-1db8c898-aded-47e7-b214-94a3520b89cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:36:38.633258 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 686/1348] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:36:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:36:38.673536 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e00e3c8-2af9-435b-addc-7bf5421647d5 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:36:38.675914 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-8e00e3c8-2af9-435b-addc-7bf5421647d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:36:38.737274 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:36:38.795043 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e00e3c8-2af9-435b-addc-7bf5421647d5 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:36:38.874581 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469f9924-51d8-43dc-add7-e53ea4c694b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76817cd541294331b0023069092e8e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:36:38.875139 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469f9924-51d8-43dc-add7-e53ea4c694b6 service neutron] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:36:38.875423 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469f9924-51d8-43dc-add7-e53ea4c694b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:36:38.875699 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469f9924-51d8-43dc-add7-e53ea4c694b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:36:38.877593 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-469f9924-51d8-43dc-add7-e53ea4c694b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:36:38.991663 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:36:38.996258 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469f9924-51d8-43dc-add7-e53ea4c694b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:36:38.997762 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469f9924-51d8-43dc-add7-e53ea4c694b6 service neutron] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:36:39.018222 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 663/1349] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:36:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:36:44.070237 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c24e2e-3bfa-46da-944e-988ab60bd3a8 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:36:44.070717 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c24e2e-3bfa-46da-944e-988ab60bd3a8 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:36:44.071223 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c24e2e-3bfa-46da-944e-988ab60bd3a8 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:36:44.073369 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 687/1350] 10.222.0.89 () {54 vars in 921 bytes} [Thu Jul 28 17:36:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:36:44.079010 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea05311-38d6-4cd4-8526-5aa28cff8ad4 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:36:44.079926 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea05311-38d6-4cd4-8526-5aa28cff8ad4 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:36:44.081047 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea05311-38d6-4cd4-8526-5aa28cff8ad4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:36:44.211957 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-9ea05311-38d6-4cd4-8526-5aa28cff8ad4 None None] MFA Rules not processed for user `e4bd290018394b0d955cd376ce35b77f`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:36:44.213479 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-9ea05311-38d6-4cd4-8526-5aa28cff8ad4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:36:44.366148 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 664/1351] 10.222.0.89 () {58 vars in 1018 bytes} [Thu Jul 28 17:36:44 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:36:48.480859 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de173f87-1ebd-40c9-85db-6bc3f0358b27 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:36:48.481495 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de173f87-1ebd-40c9-85db-6bc3f0358b27 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:36:48.481495 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de173f87-1ebd-40c9-85db-6bc3f0358b27 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:36:48.483112 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 688/1352] 10.222.0.89 () {54 vars in 921 bytes} [Thu Jul 28 17:36:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:36:48.489341 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c078200-7cd3-4701-8c83-2112904292f0 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:36:48.489854 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c078200-7cd3-4701-8c83-2112904292f0 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:36:48.489951 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c078200-7cd3-4701-8c83-2112904292f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:36:48.554732 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:36:48.581240 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-9c078200-7cd3-4701-8c83-2112904292f0 None None] MFA Rules not processed for user `e4bd290018394b0d955cd376ce35b77f`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:36:48.582477 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-9c078200-7cd3-4701-8c83-2112904292f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:36:48.631061 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=59 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:36:48.706623 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 665/1353] 10.222.0.89 () {58 vars in 1018 bytes} [Thu Jul 28 17:36:48 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:36:50.235053 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6802d842-fed0-4047-8260-63b186b476fe None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:36:50.235614 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6802d842-fed0-4047-8260-63b186b476fe None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:36:50.235614 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6802d842-fed0-4047-8260-63b186b476fe None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:36:50.237343 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 689/1354] 10.222.0.89 () {54 vars in 921 bytes} [Thu Jul 28 17:36:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:36:50.242186 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98719307-d866-4261-a282-75281835713b None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:36:50.242641 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98719307-d866-4261-a282-75281835713b None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:36:50.242740 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98719307-d866-4261-a282-75281835713b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:36:50.321441 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-98719307-d866-4261-a282-75281835713b None None] MFA Rules not processed for user `e4bd290018394b0d955cd376ce35b77f`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:36:50.322766 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-98719307-d866-4261-a282-75281835713b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:36:50.365698 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:36:50.448064 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 666/1355] 10.222.0.89 () {58 vars in 1018 bytes} [Thu Jul 28 17:36:50 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:36:53.787018 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fa0ae7-aa4e-44be-9e41-77fd8c3e832a None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:36:53.787524 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fa0ae7-aa4e-44be-9e41-77fd8c3e832a None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:36:53.787524 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fa0ae7-aa4e-44be-9e41-77fd8c3e832a None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:36:53.790302 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 690/1356] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:36:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:36:53.794263 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff41f483-668f-49ee-81f0-5a8c22fd8934 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:36:53.794709 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff41f483-668f-49ee-81f0-5a8c22fd8934 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:36:53.794825 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff41f483-668f-49ee-81f0-5a8c22fd8934 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:36:53.875014 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-ff41f483-668f-49ee-81f0-5a8c22fd8934 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:36:53.876309 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-ff41f483-668f-49ee-81f0-5a8c22fd8934 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:36:53.978845 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 667/1357] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:36:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:36:54.032609 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dee9f9-aef5-4b8b-b389-3d2a4b8bc943 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:36:54.032898 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dee9f9-aef5-4b8b-b389-3d2a4b8bc943 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:36:54.033408 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dee9f9-aef5-4b8b-b389-3d2a4b8bc943 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:36:54.034900 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 691/1358] 10.222.0.89 () {54 vars in 962 bytes} [Thu Jul 28 17:36:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:36:54.046798 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fee8233-5dba-41a7-a346-200286856f2f None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:36:54.047701 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fee8233-5dba-41a7-a346-200286856f2f None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:36:54.048393 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fee8233-5dba-41a7-a346-200286856f2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:36:54.072829 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:36:54.146392 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-8fee8233-5dba-41a7-a346-200286856f2f None None] MFA Rules not processed for user `d10b49b29df94404a71f43273839c252`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:36:54.147517 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-8fee8233-5dba-41a7-a346-200286856f2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:36:54.297242 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 668/1359] 10.222.0.89 () {58 vars in 1059 bytes} [Thu Jul 28 17:36:54 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:36:54.305643 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dee9f9-aef5-4b8b-b389-3d2a4b8bc943 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:36:54.307241 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-56dee9f9-aef5-4b8b-b389-3d2a4b8bc943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:36:54.462949 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dee9f9-aef5-4b8b-b389-3d2a4b8bc943 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:36:54.554003 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:36:54.603482 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de87db56-b689-4b20-ae52-2e58571b6a68 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd10b49b29df94404a71f43273839c252', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:36:54.604099 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de87db56-b689-4b20-ae52-2e58571b6a68 service nova] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:36:54.604383 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de87db56-b689-4b20-ae52-2e58571b6a68 service nova] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:36:54.604661 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de87db56-b689-4b20-ae52-2e58571b6a68 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:36:54.606351 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-de87db56-b689-4b20-ae52-2e58571b6a68 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:36:54.704987 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de87db56-b689-4b20-ae52-2e58571b6a68 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:36:54.707164 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de87db56-b689-4b20-ae52-2e58571b6a68 service nova] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:36:54.728706 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 692/1360] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:36:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:36:57.005763 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7f184c-eb51-4d91-9b45-6bdeca7f4814 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:36:57.006288 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7f184c-eb51-4d91-9b45-6bdeca7f4814 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:36:57.006395 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7f184c-eb51-4d91-9b45-6bdeca7f4814 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:36:57.008085 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 669/1361] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:36:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:36:57.013610 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a702de4-9fc8-414b-9eb3-f5e4c5d4ae56 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:36:57.013610 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a702de4-9fc8-414b-9eb3-f5e4c5d4ae56 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:36:57.013610 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a702de4-9fc8-414b-9eb3-f5e4c5d4ae56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:36:57.085310 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-9a702de4-9fc8-414b-9eb3-f5e4c5d4ae56 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:36:57.086501 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-9a702de4-9fc8-414b-9eb3-f5e4c5d4ae56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:36:57.131281 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:36:57.213782 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 693/1362] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:36:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:36:57.259470 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7f184c-eb51-4d91-9b45-6bdeca7f4814 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:36:57.261069 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-3b7f184c-eb51-4d91-9b45-6bdeca7f4814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:36:57.371096 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:36:57.454761 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7f184c-eb51-4d91-9b45-6bdeca7f4814 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:36:57.593788 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a215a2-36ee-4a36-a11b-f71189b125b7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd10b49b29df94404a71f43273839c252', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:36:57.594486 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a215a2-36ee-4a36-a11b-f71189b125b7 service nova] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:36:57.594687 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a215a2-36ee-4a36-a11b-f71189b125b7 service nova] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:36:57.594952 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a215a2-36ee-4a36-a11b-f71189b125b7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:36:57.597811 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-95a215a2-36ee-4a36-a11b-f71189b125b7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:36:57.622166 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:36:57.695567 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a215a2-36ee-4a36-a11b-f71189b125b7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:36:57.697189 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a215a2-36ee-4a36-a11b-f71189b125b7 service nova] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:36:57.719640 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 670/1363] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:36:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:37:00.085918 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2931528-1269-45a8-a3b1-8fc5f0553b8b None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:37:00.086780 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2931528-1269-45a8-a3b1-8fc5f0553b8b None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:37:00.087206 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2931528-1269-45a8-a3b1-8fc5f0553b8b None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:37:00.088985 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 694/1364] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:37:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:37:00.094842 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5180c49-f913-4836-a16a-2098f9cbc5d4 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:37:00.095340 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5180c49-f913-4836-a16a-2098f9cbc5d4 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:37:00.095456 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5180c49-f913-4836-a16a-2098f9cbc5d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:37:00.166885 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-b5180c49-f913-4836-a16a-2098f9cbc5d4 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:37:00.168100 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-b5180c49-f913-4836-a16a-2098f9cbc5d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:37:00.281271 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:37:00.281271 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 671/1365] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:37:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:37:00.322039 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db17256-0fec-4b99-9604-7f359aa7880f None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:37:00.322605 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db17256-0fec-4b99-9604-7f359aa7880f None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:37:00.323541 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db17256-0fec-4b99-9604-7f359aa7880f None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:37:00.325017 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 695/1366] 10.222.0.89 () {54 vars in 962 bytes} [Thu Jul 28 17:37:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:37:00.331825 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5ab3bc-cb3d-47f6-b211-afe983ec439d None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:37:00.332076 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5ab3bc-cb3d-47f6-b211-afe983ec439d None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:37:00.332326 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5ab3bc-cb3d-47f6-b211-afe983ec439d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:37:00.412151 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-bf5ab3bc-cb3d-47f6-b211-afe983ec439d None None] MFA Rules not processed for user `d10b49b29df94404a71f43273839c252`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:37:00.413315 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-bf5ab3bc-cb3d-47f6-b211-afe983ec439d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:37:00.542409 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 672/1367] 10.222.0.89 () {58 vars in 1059 bytes} [Thu Jul 28 17:37:00 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:37:00.551031 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db17256-0fec-4b99-9604-7f359aa7880f None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:37:00.552875 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-8db17256-0fec-4b99-9604-7f359aa7880f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:37:00.710045 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db17256-0fec-4b99-9604-7f359aa7880f None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:37:00.737507 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:37:00.856954 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75cdf07-1b66-4673-9884-c7436c338ad7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd10b49b29df94404a71f43273839c252', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:37:00.858080 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75cdf07-1b66-4673-9884-c7436c338ad7 service nova] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:37:00.858536 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75cdf07-1b66-4673-9884-c7436c338ad7 service nova] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:37:00.860000 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75cdf07-1b66-4673-9884-c7436c338ad7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:37:00.862873 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-f75cdf07-1b66-4673-9884-c7436c338ad7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:37:00.968918 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75cdf07-1b66-4673-9884-c7436c338ad7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:37:00.970601 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75cdf07-1b66-4673-9884-c7436c338ad7 service nova] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:37:00.990407 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 696/1368] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:37:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:37:01.944719 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13657dc8-babe-43e2-ac0f-b758bffe3686 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:37:01.945510 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13657dc8-babe-43e2-ac0f-b758bffe3686 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:37:01.945510 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13657dc8-babe-43e2-ac0f-b758bffe3686 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:37:01.946940 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 673/1369] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:37:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:37:01.952481 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57edae6a-59eb-4266-b18a-e3bd310f8b87 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:37:01.952917 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57edae6a-59eb-4266-b18a-e3bd310f8b87 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:37:01.953082 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57edae6a-59eb-4266-b18a-e3bd310f8b87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:37:01.987829 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:37:02.026141 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-57edae6a-59eb-4266-b18a-e3bd310f8b87 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:37:02.027466 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-57edae6a-59eb-4266-b18a-e3bd310f8b87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:37:02.136666 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 697/1370] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:37:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:37:02.486206 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc56d7e-a9b3-487b-8290-8ac1c664f4ed None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:37:02.486206 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc56d7e-a9b3-487b-8290-8ac1c664f4ed None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:37:02.486808 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc56d7e-a9b3-487b-8290-8ac1c664f4ed None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:37:02.489521 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 674/1371] 10.222.0.89 () {54 vars in 964 bytes} [Thu Jul 28 17:37:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:37:02.494245 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75b69ce-9225-48ba-b946-625a1ae31337 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:37:02.494866 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75b69ce-9225-48ba-b946-625a1ae31337 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:37:02.495355 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75b69ce-9225-48ba-b946-625a1ae31337 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:37:02.582191 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-e75b69ce-9225-48ba-b946-625a1ae31337 None None] MFA Rules not processed for user `96d512d0c84240918dae6c4a3e8b7b79`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:37:02.595002 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-e75b69ce-9225-48ba-b946-625a1ae31337 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:37:02.647350 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:37:02.681966 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 698/1372] 10.222.0.89 () {58 vars in 1061 bytes} [Thu Jul 28 17:37:02 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:37:02.689370 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc56d7e-a9b3-487b-8290-8ac1c664f4ed None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:37:02.691085 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-3bc56d7e-a9b3-487b-8290-8ac1c664f4ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:37:02.749952 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:37:02.810933 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc56d7e-a9b3-487b-8290-8ac1c664f4ed None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:37:02.895105 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89322f0-e246-4ba6-a294-1feeb45e8bee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96d512d0c84240918dae6c4a3e8b7b79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:37:02.895849 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89322f0-e246-4ba6-a294-1feeb45e8bee service cinder] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:37:02.896132 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89322f0-e246-4ba6-a294-1feeb45e8bee service cinder] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:37:02.896412 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89322f0-e246-4ba6-a294-1feeb45e8bee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:37:02.898181 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-d89322f0-e246-4ba6-a294-1feeb45e8bee service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:37:03.024292 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:37:03.046793 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89322f0-e246-4ba6-a294-1feeb45e8bee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:37:03.048874 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89322f0-e246-4ba6-a294-1feeb45e8bee service cinder] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:37:03.078284 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 675/1373] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:37:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:37:03.151025 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5db6d72-8f5d-4c4b-9184-c0ee52d33aef None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:37:03.151949 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5db6d72-8f5d-4c4b-9184-c0ee52d33aef None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:37:03.151949 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5db6d72-8f5d-4c4b-9184-c0ee52d33aef None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:37:03.157413 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 699/1374] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:37:03 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:37:03.158347 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efe9bdd-28dc-4dcd-bc77-57b3fc8d9004 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:37:03.159398 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efe9bdd-28dc-4dcd-bc77-57b3fc8d9004 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:37:03.159874 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efe9bdd-28dc-4dcd-bc77-57b3fc8d9004 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:37:03.230653 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-2efe9bdd-28dc-4dcd-bc77-57b3fc8d9004 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:37:03.231839 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-2efe9bdd-28dc-4dcd-bc77-57b3fc8d9004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:37:03.336255 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 676/1375] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:37:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:37:03.383654 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5db6d72-8f5d-4c4b-9184-c0ee52d33aef None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:37:03.385184 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-a5db6d72-8f5d-4c4b-9184-c0ee52d33aef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:37:03.537239 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5db6d72-8f5d-4c4b-9184-c0ee52d33aef None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:37:03.592410 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:37:03.655960 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739c7c6e-d55f-4fed-bff6-e9640116ed26 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd10b49b29df94404a71f43273839c252', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:37:03.656575 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739c7c6e-d55f-4fed-bff6-e9640116ed26 service nova] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:37:03.656853 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739c7c6e-d55f-4fed-bff6-e9640116ed26 service nova] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:37:03.657124 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739c7c6e-d55f-4fed-bff6-e9640116ed26 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:37:03.659750 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-739c7c6e-d55f-4fed-bff6-e9640116ed26 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:37:03.753796 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739c7c6e-d55f-4fed-bff6-e9640116ed26 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:37:03.755391 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739c7c6e-d55f-4fed-bff6-e9640116ed26 service nova] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:37:03.774874 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 700/1376] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:37:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:37:05.879148 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb2615f-5dba-4a9b-849c-05e0bf13bf3e None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:37:05.879687 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb2615f-5dba-4a9b-849c-05e0bf13bf3e None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:37:05.879687 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb2615f-5dba-4a9b-849c-05e0bf13bf3e None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:37:05.881059 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 677/1377] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:37:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:37:05.886437 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d1a933-94b4-4b8a-930a-47360b4170eb None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:37:05.886844 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d1a933-94b4-4b8a-930a-47360b4170eb None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:37:05.886988 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d1a933-94b4-4b8a-930a-47360b4170eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:37:05.958222 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-b2d1a933-94b4-4b8a-930a-47360b4170eb None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:37:05.959472 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-b2d1a933-94b4-4b8a-930a-47360b4170eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:37:05.966406 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:37:06.066489 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 701/1378] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:37:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:37:06.111038 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb2615f-5dba-4a9b-849c-05e0bf13bf3e None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:37:06.113228 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-4bb2615f-5dba-4a9b-849c-05e0bf13bf3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:37:06.199953 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:37:06.288892 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb2615f-5dba-4a9b-849c-05e0bf13bf3e None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:37:06.404602 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbec9a2-b212-4795-9f70-6d85532f71e0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd10b49b29df94404a71f43273839c252', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:37:06.405142 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbec9a2-b212-4795-9f70-6d85532f71e0 service nova] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:37:06.405475 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbec9a2-b212-4795-9f70-6d85532f71e0 service nova] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:37:06.405745 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbec9a2-b212-4795-9f70-6d85532f71e0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:37:06.407509 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-5fbec9a2-b212-4795-9f70-6d85532f71e0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:37:06.427281 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:37:06.520776 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbec9a2-b212-4795-9f70-6d85532f71e0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:37:06.522536 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbec9a2-b212-4795-9f70-6d85532f71e0 service nova] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:37:06.543006 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 678/1379] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:37:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:37:08.671172 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066740d8-7a6e-4b20-a4fb-00907ec52498 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:37:08.671730 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066740d8-7a6e-4b20-a4fb-00907ec52498 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:37:08.671730 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066740d8-7a6e-4b20-a4fb-00907ec52498 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:37:08.673373 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 702/1380] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:37:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:37:08.679366 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73e76ea-3270-4349-9eae-1349f25586ea None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:37:08.679952 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73e76ea-3270-4349-9eae-1349f25586ea None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:37:08.679952 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73e76ea-3270-4349-9eae-1349f25586ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:37:08.766068 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-c73e76ea-3270-4349-9eae-1349f25586ea None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:37:08.767313 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-c73e76ea-3270-4349-9eae-1349f25586ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:37:08.879936 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:37:08.888917 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 679/1381] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:37:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 17:37:08.933954 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066740d8-7a6e-4b20-a4fb-00907ec52498 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:37:08.935605 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-066740d8-7a6e-4b20-a4fb-00907ec52498 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:37:09.098180 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:37:09.114182 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066740d8-7a6e-4b20-a4fb-00907ec52498 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:37:09.244417 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1a7fc5-d657-4fe1-93fe-fa962a4cbb14 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd10b49b29df94404a71f43273839c252', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:37:09.245211 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1a7fc5-d657-4fe1-93fe-fa962a4cbb14 service nova] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:37:09.245610 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1a7fc5-d657-4fe1-93fe-fa962a4cbb14 service nova] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:37:09.245974 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1a7fc5-d657-4fe1-93fe-fa962a4cbb14 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:37:09.247988 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-da1a7fc5-d657-4fe1-93fe-fa962a4cbb14 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:37:09.352742 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1a7fc5-d657-4fe1-93fe-fa962a4cbb14 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:37:09.354526 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1a7fc5-d657-4fe1-93fe-fa962a4cbb14 service nova] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:37:09.367229 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:37:09.376784 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 703/1382] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:37:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:37:11.505796 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838d24f5-4c7e-4d70-98f5-d8a45739323c None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:37:11.506473 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838d24f5-4c7e-4d70-98f5-d8a45739323c None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:37:11.506473 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838d24f5-4c7e-4d70-98f5-d8a45739323c None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:37:11.508031 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 680/1383] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:37:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:37:11.513585 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94e1a67-d4ad-4d04-8d1b-913ea2bf76cd None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:37:11.514114 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94e1a67-d4ad-4d04-8d1b-913ea2bf76cd None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:37:11.514196 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94e1a67-d4ad-4d04-8d1b-913ea2bf76cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:37:11.584365 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-e94e1a67-d4ad-4d04-8d1b-913ea2bf76cd None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:37:11.585640 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-e94e1a67-d4ad-4d04-8d1b-913ea2bf76cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:37:11.687933 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 704/1384] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:37:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:37:11.730519 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838d24f5-4c7e-4d70-98f5-d8a45739323c None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:37:11.732105 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-838d24f5-4c7e-4d70-98f5-d8a45739323c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:37:11.907608 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838d24f5-4c7e-4d70-98f5-d8a45739323c None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:37:11.979470 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:37:12.029496 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ae9033-f47f-481d-ad9c-593926606e43 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd10b49b29df94404a71f43273839c252', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:37:12.030109 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ae9033-f47f-481d-ad9c-593926606e43 service nova] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:37:12.030382 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ae9033-f47f-481d-ad9c-593926606e43 service nova] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:37:12.030650 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ae9033-f47f-481d-ad9c-593926606e43 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:37:12.032274 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-a8ae9033-f47f-481d-ad9c-593926606e43 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:37:12.131002 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ae9033-f47f-481d-ad9c-593926606e43 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:37:12.132526 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ae9033-f47f-481d-ad9c-593926606e43 service nova] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:37:12.152057 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 681/1385] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:37:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:37:14.217662 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fa0e51-342b-4147-9b6e-0f495d76f6f4 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:37:14.218276 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fa0e51-342b-4147-9b6e-0f495d76f6f4 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:37:14.218276 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fa0e51-342b-4147-9b6e-0f495d76f6f4 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:37:14.219565 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 705/1386] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:37:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:37:14.225019 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500a8e31-a322-49c6-8f30-fc7f21a566c4 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:37:14.225652 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500a8e31-a322-49c6-8f30-fc7f21a566c4 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:37:14.225652 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500a8e31-a322-49c6-8f30-fc7f21a566c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:37:14.302202 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-500a8e31-a322-49c6-8f30-fc7f21a566c4 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:37:14.303860 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-500a8e31-a322-49c6-8f30-fc7f21a566c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:37:14.346307 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:37:14.413810 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 682/1387] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:37:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:37:14.459240 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fa0e51-342b-4147-9b6e-0f495d76f6f4 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:37:14.461274 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-18fa0e51-342b-4147-9b6e-0f495d76f6f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:37:14.555061 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:37:14.642122 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fa0e51-342b-4147-9b6e-0f495d76f6f4 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:37:14.761652 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad619cd4-ebef-488a-8bdf-a109f2cba624 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd10b49b29df94404a71f43273839c252', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:37:14.762265 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad619cd4-ebef-488a-8bdf-a109f2cba624 service nova] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:37:14.762571 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad619cd4-ebef-488a-8bdf-a109f2cba624 service nova] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:37:14.762815 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad619cd4-ebef-488a-8bdf-a109f2cba624 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:37:14.764428 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-ad619cd4-ebef-488a-8bdf-a109f2cba624 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:37:14.807147 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:37:14.866460 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad619cd4-ebef-488a-8bdf-a109f2cba624 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:37:14.868112 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad619cd4-ebef-488a-8bdf-a109f2cba624 service nova] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:37:14.887962 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 706/1388] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:37:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:37:17.134469 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a0a8d3-96cc-470f-b799-af1350b001e3 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:37:17.135033 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a0a8d3-96cc-470f-b799-af1350b001e3 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:37:17.135033 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a0a8d3-96cc-470f-b799-af1350b001e3 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:37:17.136274 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 683/1389] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:37:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:37:17.141380 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdc76a6-4001-4c87-8d3c-94c07af53dc4 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:37:17.141843 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdc76a6-4001-4c87-8d3c-94c07af53dc4 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:37:17.141952 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdc76a6-4001-4c87-8d3c-94c07af53dc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:37:17.208484 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-dcdc76a6-4001-4c87-8d3c-94c07af53dc4 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:37:17.209716 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-dcdc76a6-4001-4c87-8d3c-94c07af53dc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:37:17.307447 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 707/1390] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:37:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:37:17.352414 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a0a8d3-96cc-470f-b799-af1350b001e3 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:37:17.353964 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-69a0a8d3-96cc-470f-b799-af1350b001e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:37:17.503700 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a0a8d3-96cc-470f-b799-af1350b001e3 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:37:17.520357 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:37:17.626025 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc22e279-81a6-499b-aba2-feef0ab90a33 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd10b49b29df94404a71f43273839c252', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:37:17.626655 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc22e279-81a6-499b-aba2-feef0ab90a33 service nova] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:37:17.626934 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc22e279-81a6-499b-aba2-feef0ab90a33 service nova] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:37:17.627209 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc22e279-81a6-499b-aba2-feef0ab90a33 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:37:17.628948 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-bc22e279-81a6-499b-aba2-feef0ab90a33 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:37:17.720341 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc22e279-81a6-499b-aba2-feef0ab90a33 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:37:17.722539 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc22e279-81a6-499b-aba2-feef0ab90a33 service nova] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:37:17.741264 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 684/1391] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:37:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:37:18.526013 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155ce505-dcf6-4aa0-8adc-668d0feb38f9 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:37:18.526584 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155ce505-dcf6-4aa0-8adc-668d0feb38f9 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:37:18.526584 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155ce505-dcf6-4aa0-8adc-668d0feb38f9 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:37:18.527917 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 708/1392] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:37:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:37:18.534569 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f3713f-d4ea-4afd-9bf9-fd3711e01e0f None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:37:18.535180 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f3713f-d4ea-4afd-9bf9-fd3711e01e0f None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:37:18.535180 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f3713f-d4ea-4afd-9bf9-fd3711e01e0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:37:18.562427 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:37:18.603875 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-38f3713f-d4ea-4afd-9bf9-fd3711e01e0f None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:37:18.605094 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-38f3713f-d4ea-4afd-9bf9-fd3711e01e0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:37:18.702176 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 685/1393] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:37:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:37:18.759826 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ddc384-89ae-46cb-b695-93c5f66994b2 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:37:18.760110 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ddc384-89ae-46cb-b695-93c5f66994b2 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:37:18.760382 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ddc384-89ae-46cb-b695-93c5f66994b2 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:37:18.761942 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 709/1394] 10.222.0.89 () {54 vars in 966 bytes} [Thu Jul 28 17:37:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:37:18.767089 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3014a22-bfba-43f0-b561-82f3ad0c1193 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:37:18.767373 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3014a22-bfba-43f0-b561-82f3ad0c1193 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:37:18.767648 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3014a22-bfba-43f0-b561-82f3ad0c1193 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:37:18.849198 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-d3014a22-bfba-43f0-b561-82f3ad0c1193 None None] MFA Rules not processed for user `1564db0771224a05b47d8e3cf778c147`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:37:18.850411 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-d3014a22-bfba-43f0-b561-82f3ad0c1193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:37:18.896633 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:37:18.935226 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 686/1395] 10.222.0.89 () {58 vars in 1063 bytes} [Thu Jul 28 17:37:18 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:37:18.943055 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ddc384-89ae-46cb-b695-93c5f66994b2 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:37:18.944638 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-94ddc384-89ae-46cb-b695-93c5f66994b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:37:18.959004 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:37:19.050792 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ddc384-89ae-46cb-b695-93c5f66994b2 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:37:19.123602 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429c3cbb-16c3-43b0-b767-b273d570f29b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1564db0771224a05b47d8e3cf778c147', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:37:19.124130 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429c3cbb-16c3-43b0-b767-b273d570f29b service glance] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:37:19.124411 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429c3cbb-16c3-43b0-b767-b273d570f29b service glance] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:37:19.124692 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429c3cbb-16c3-43b0-b767-b273d570f29b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:37:19.126336 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-429c3cbb-16c3-43b0-b767-b273d570f29b service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:37:19.195699 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:37:19.238846 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429c3cbb-16c3-43b0-b767-b273d570f29b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:37:19.240279 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429c3cbb-16c3-43b0-b767-b273d570f29b service glance] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:37:19.259555 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 710/1396] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:37:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:37:19.819187 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8864d173-b7ba-499e-9c7c-64551c23d498 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:37:19.819744 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8864d173-b7ba-499e-9c7c-64551c23d498 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:37:19.819913 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8864d173-b7ba-499e-9c7c-64551c23d498 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:37:19.821678 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 687/1397] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:37:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:37:19.826993 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487e56a5-1de8-4091-9ead-ddb81e2c4513 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:37:19.827448 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487e56a5-1de8-4091-9ead-ddb81e2c4513 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:37:19.827750 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487e56a5-1de8-4091-9ead-ddb81e2c4513 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:37:19.906981 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-487e56a5-1de8-4091-9ead-ddb81e2c4513 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:37:19.908231 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-487e56a5-1de8-4091-9ead-ddb81e2c4513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:37:20.016028 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 711/1398] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:37:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:37:20.018244 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5373bfb8-c62b-4b12-a5d1-fad8cd55d9ad None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:37:20.018662 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5373bfb8-c62b-4b12-a5d1-fad8cd55d9ad None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:37:20.019112 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5373bfb8-c62b-4b12-a5d1-fad8cd55d9ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:37:20.061127 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487e56a5-1de8-4091-9ead-ddb81e2c4513 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:37:20.062631 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-487e56a5-1de8-4091-9ead-ddb81e2c4513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:37:20.093829 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:37:20.194939 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487e56a5-1de8-4091-9ead-ddb81e2c4513 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:37:20.221867 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-5373bfb8-c62b-4b12-a5d1-fad8cd55d9ad None None] MFA Rules not processed for user `c44387d13a694ee6bb47a13906003330`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:37:20.223605 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-5373bfb8-c62b-4b12-a5d1-fad8cd55d9ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:37:20.314984 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 688/1399] 10.222.0.89 () {58 vars in 1009 bytes} [Thu Jul 28 17:37:20 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:37:20.316019 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b085301-f59d-4ff8-844d-90e791ad0de6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd10b49b29df94404a71f43273839c252', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:37:20.316795 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b085301-f59d-4ff8-844d-90e791ad0de6 service nova] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:37:20.317086 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b085301-f59d-4ff8-844d-90e791ad0de6 service nova] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:37:20.317319 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b085301-f59d-4ff8-844d-90e791ad0de6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:37:20.318997 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-9b085301-f59d-4ff8-844d-90e791ad0de6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:37:20.338398 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:37:20.366954 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a914d0-a7f4-4349-8280-6ad69037ad10 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:37:20.367317 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a914d0-a7f4-4349-8280-6ad69037ad10 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:37:20.367659 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a914d0-a7f4-4349-8280-6ad69037ad10 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:37:20.369035 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 689/1400] 10.222.0.89 () {54 vars in 943 bytes} [Thu Jul 28 17:37:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 17:37:20.375207 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25183bb7-6c40-493e-a09c-3deed26f2b4a None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:37:20.375484 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25183bb7-6c40-493e-a09c-3deed26f2b4a None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:37:20.375750 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25183bb7-6c40-493e-a09c-3deed26f2b4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:37:20.416471 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b085301-f59d-4ff8-844d-90e791ad0de6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:37:20.418189 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b085301-f59d-4ff8-844d-90e791ad0de6 service nova] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:37:20.438510 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 712/1401] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:37:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 17:37:20.452971 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:37:20.464540 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-25183bb7-6c40-493e-a09c-3deed26f2b4a None None] MFA Rules not processed for user `3c59473b01bf4433ba3b111114fc048b`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:37:20.465802 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-25183bb7-6c40-493e-a09c-3deed26f2b4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:37:20.589244 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 690/1402] 10.222.0.89 () {58 vars in 1040 bytes} [Thu Jul 28 17:37:20 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 17:37:20.599691 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b085301-f59d-4ff8-844d-90e791ad0de6 service nova] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:37:20.600917 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-9b085301-f59d-4ff8-844d-90e791ad0de6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:37:20.694757 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b085301-f59d-4ff8-844d-90e791ad0de6 service nova] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:37:20.752850 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:37:20.803893 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ed2227-7e0a-48e7-8ed9-6fadd3589d87 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c59473b01bf4433ba3b111114fc048b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:37:20.804509 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ed2227-7e0a-48e7-8ed9-6fadd3589d87 service swift] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:37:20.804788 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ed2227-7e0a-48e7-8ed9-6fadd3589d87 service swift] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:37:20.805055 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ed2227-7e0a-48e7-8ed9-6fadd3589d87 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:37:20.806718 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-21ed2227-7e0a-48e7-8ed9-6fadd3589d87 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:37:20.896319 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ed2227-7e0a-48e7-8ed9-6fadd3589d87 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:37:20.897856 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ed2227-7e0a-48e7-8ed9-6fadd3589d87 service swift] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:37:20.906019 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 713/1403] 10.222.0.89 () {60 vars in 1384 bytes} [Thu Jul 28 17:37:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 17:37:22.501130 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1ff774-88e7-4041-98c2-9a47b56ca778 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:37:22.501724 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1ff774-88e7-4041-98c2-9a47b56ca778 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:37:22.501724 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1ff774-88e7-4041-98c2-9a47b56ca778 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:37:22.503213 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 691/1404] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:37:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:37:22.509038 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b32cb55-2fa9-4f0f-aa30-ad8e326d0bed None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:37:22.509526 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b32cb55-2fa9-4f0f-aa30-ad8e326d0bed None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:37:22.509722 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b32cb55-2fa9-4f0f-aa30-ad8e326d0bed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:37:22.591780 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-5b32cb55-2fa9-4f0f-aa30-ad8e326d0bed None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:37:22.593191 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-5b32cb55-2fa9-4f0f-aa30-ad8e326d0bed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:37:22.643072 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:37:22.709968 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 714/1405] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:37:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:37:22.757314 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1ff774-88e7-4041-98c2-9a47b56ca778 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:37:22.759021 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-0d1ff774-88e7-4041-98c2-9a47b56ca778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:37:22.905862 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:37:22.951564 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1ff774-88e7-4041-98c2-9a47b56ca778 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:37:23.090362 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc81231-9179-4a82-8d29-424852021225 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd10b49b29df94404a71f43273839c252', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:37:23.090921 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc81231-9179-4a82-8d29-424852021225 service nova] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:37:23.091203 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc81231-9179-4a82-8d29-424852021225 service nova] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:37:23.091494 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc81231-9179-4a82-8d29-424852021225 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:37:23.093119 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-7dc81231-9179-4a82-8d29-424852021225 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:37:23.204870 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:37:23.221174 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc81231-9179-4a82-8d29-424852021225 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:37:23.222968 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc81231-9179-4a82-8d29-424852021225 service nova] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:37:23.243195 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 692/1406] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:37:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 486 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:37:25.405682 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b30801a-a342-4823-b872-3689529c1700 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:37:25.406321 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b30801a-a342-4823-b872-3689529c1700 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:37:25.406321 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b30801a-a342-4823-b872-3689529c1700 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:37:25.411350 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 715/1407] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:37:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:37:25.413460 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a504e8c-969b-4782-825f-d2cb46f57714 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:37:25.413907 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a504e8c-969b-4782-825f-d2cb46f57714 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:37:25.414001 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a504e8c-969b-4782-825f-d2cb46f57714 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:37:25.494637 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-6a504e8c-969b-4782-825f-d2cb46f57714 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:37:25.496116 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-6a504e8c-969b-4782-825f-d2cb46f57714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:37:25.608500 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 693/1408] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:37:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:37:25.655508 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b30801a-a342-4823-b872-3689529c1700 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:37:25.657272 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-6b30801a-a342-4823-b872-3689529c1700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:37:25.841542 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b30801a-a342-4823-b872-3689529c1700 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:37:25.871015 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:37:25.972755 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9ecd88-2622-483e-a9a2-b8890a758b4d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd10b49b29df94404a71f43273839c252', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:37:25.974325 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9ecd88-2622-483e-a9a2-b8890a758b4d service nova] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:37:25.974760 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9ecd88-2622-483e-a9a2-b8890a758b4d service nova] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:37:25.975044 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9ecd88-2622-483e-a9a2-b8890a758b4d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:37:25.976900 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-7b9ecd88-2622-483e-a9a2-b8890a758b4d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:37:26.079107 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9ecd88-2622-483e-a9a2-b8890a758b4d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:37:26.081391 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9ecd88-2622-483e-a9a2-b8890a758b4d service nova] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:37:26.101769 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 716/1409] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:37:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:37:35.606432 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=76 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:37:36.099432 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=87 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:37:43.100735 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf3c46a-8b49-4494-8d0d-57bc20e3148e None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:37:43.101303 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf3c46a-8b49-4494-8d0d-57bc20e3148e None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:37:43.101303 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf3c46a-8b49-4494-8d0d-57bc20e3148e None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:37:43.102592 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 694/1410] 10.222.0.89 () {54 vars in 918 bytes} [Thu Jul 28 17:37:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:37:43.107697 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73005ac-44b6-4536-b5d2-82c77ab5db25 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:37:43.108118 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73005ac-44b6-4536-b5d2-82c77ab5db25 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:37:43.108244 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73005ac-44b6-4536-b5d2-82c77ab5db25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:37:43.183082 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-f73005ac-44b6-4536-b5d2-82c77ab5db25 None None] MFA Rules not processed for user `e4bd290018394b0d955cd376ce35b77f`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:37:43.184297 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-f73005ac-44b6-4536-b5d2-82c77ab5db25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:37:43.290173 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 717/1411] 10.222.0.89 () {58 vars in 1015 bytes} [Thu Jul 28 17:37:43 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:37:43.305062 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec626220-cd8f-4607-8cb8-a43bad24143f None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:37:43.305444 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec626220-cd8f-4607-8cb8-a43bad24143f None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:37:43.305718 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec626220-cd8f-4607-8cb8-a43bad24143f None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:37:43.307049 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 695/1412] 10.222.0.89 () {54 vars in 918 bytes} [Thu Jul 28 17:37:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:37:43.311658 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73878595-e094-49ac-b429-0bbef3a97a2f None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:37:43.311932 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73878595-e094-49ac-b429-0bbef3a97a2f None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:37:43.312213 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73878595-e094-49ac-b429-0bbef3a97a2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:37:43.385846 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:37:43.388926 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-73878595-e094-49ac-b429-0bbef3a97a2f None None] MFA Rules not processed for user `d10b49b29df94404a71f43273839c252`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:37:43.390518 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-73878595-e094-49ac-b429-0bbef3a97a2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:37:43.517777 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 718/1413] 10.222.0.89 () {58 vars in 1015 bytes} [Thu Jul 28 17:37:43 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:37:46.006781 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b02c26-4c2e-40cc-a712-443dcca37770 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:37:46.007338 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b02c26-4c2e-40cc-a712-443dcca37770 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:37:46.007338 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b02c26-4c2e-40cc-a712-443dcca37770 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:37:46.008963 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 696/1414] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 17:37:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:37:46.014515 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d027df9-5e90-4753-87eb-a3428e0c166d None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:37:46.014951 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d027df9-5e90-4753-87eb-a3428e0c166d None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:37:46.015064 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d027df9-5e90-4753-87eb-a3428e0c166d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:37:46.082172 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-5d027df9-5e90-4753-87eb-a3428e0c166d None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:37:46.083323 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-5d027df9-5e90-4753-87eb-a3428e0c166d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:37:46.139611 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:37:46.182715 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 719/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 17:37:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 17:37:46.225044 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b02c26-4c2e-40cc-a712-443dcca37770 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:37:46.226586 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-56b02c26-4c2e-40cc-a712-443dcca37770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:37:46.372518 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b02c26-4c2e-40cc-a712-443dcca37770 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:37:46.444728 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:37:46.486712 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973c6063-307b-474c-84e3-3eea31cc6d86 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd10b49b29df94404a71f43273839c252', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:37:46.487323 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973c6063-307b-474c-84e3-3eea31cc6d86 service nova] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:37:46.487600 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973c6063-307b-474c-84e3-3eea31cc6d86 service nova] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:37:46.487899 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973c6063-307b-474c-84e3-3eea31cc6d86 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:37:46.489548 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-973c6063-307b-474c-84e3-3eea31cc6d86 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:37:46.578299 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973c6063-307b-474c-84e3-3eea31cc6d86 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:37:46.580387 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973c6063-307b-474c-84e3-3eea31cc6d86 service nova] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:37:46.598812 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 697/1416] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:37:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:37:47.518178 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97651a7-dd90-461a-a132-cf61102ddf15 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:37:47.518778 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97651a7-dd90-461a-a132-cf61102ddf15 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:37:47.519128 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97651a7-dd90-461a-a132-cf61102ddf15 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:37:47.521590 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 720/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 28 17:37:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:37:47.526757 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b604c37-e85a-4297-95de-745c29840a78 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:37:47.527236 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b604c37-e85a-4297-95de-745c29840a78 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:37:47.527414 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b604c37-e85a-4297-95de-745c29840a78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:37:47.605013 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-3b604c37-e85a-4297-95de-745c29840a78 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:37:47.606330 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-3b604c37-e85a-4297-95de-745c29840a78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:37:47.635216 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:37:47.709205 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 698/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 28 17:37:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:37:47.879328 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97651a7-dd90-461a-a132-cf61102ddf15 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:37:47.880904 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-b97651a7-dd90-461a-a132-cf61102ddf15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:37:48.036339 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97651a7-dd90-461a-a132-cf61102ddf15 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:37:48.077468 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:37:48.159062 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dab36eb-3f07-4100-adcb-8749b1863ea8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd10b49b29df94404a71f43273839c252', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:37:48.159612 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dab36eb-3f07-4100-adcb-8749b1863ea8 service nova] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:37:48.159892 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dab36eb-3f07-4100-adcb-8749b1863ea8 service nova] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:37:48.160163 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dab36eb-3f07-4100-adcb-8749b1863ea8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:37:48.161855 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-6dab36eb-3f07-4100-adcb-8749b1863ea8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:37:48.263130 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dab36eb-3f07-4100-adcb-8749b1863ea8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:37:48.264714 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dab36eb-3f07-4100-adcb-8749b1863ea8 service nova] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:37:48.292854 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 721/1419] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:37:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 17:37:49.182665 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74435d6-e5d4-4794-97e2-e09ee2bbbc90 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:37:49.183217 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74435d6-e5d4-4794-97e2-e09ee2bbbc90 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:37:49.183217 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74435d6-e5d4-4794-97e2-e09ee2bbbc90 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:37:49.184574 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 699/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 28 17:37:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:37:49.189653 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3879b5d-fb90-424d-9064-8633dd9b7adf None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:37:49.190116 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3879b5d-fb90-424d-9064-8633dd9b7adf None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:37:49.190235 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3879b5d-fb90-424d-9064-8633dd9b7adf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:37:49.245124 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:37:49.267353 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-b3879b5d-fb90-424d-9064-8633dd9b7adf None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:37:49.268663 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-b3879b5d-fb90-424d-9064-8633dd9b7adf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:37:49.365867 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 722/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 28 17:37:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:37:49.518617 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74435d6-e5d4-4794-97e2-e09ee2bbbc90 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:37:49.520077 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-b74435d6-e5d4-4794-97e2-e09ee2bbbc90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:37:49.667070 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74435d6-e5d4-4794-97e2-e09ee2bbbc90 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:37:49.679275 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:37:49.782114 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aeef935-2f73-4abb-8413-7c4af07eccc3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd10b49b29df94404a71f43273839c252', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:37:49.782677 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeef935-2f73-4abb-8413-7c4af07eccc3 service nova] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:37:49.782957 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeef935-2f73-4abb-8413-7c4af07eccc3 service nova] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:37:49.783227 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeef935-2f73-4abb-8413-7c4af07eccc3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:37:49.784844 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-6aeef935-2f73-4abb-8413-7c4af07eccc3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:37:49.878524 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aeef935-2f73-4abb-8413-7c4af07eccc3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:37:49.880121 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aeef935-2f73-4abb-8413-7c4af07eccc3 service nova] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:37:49.898997 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 700/1422] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:37:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:37:50.763570 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a331b3e5-85cf-49a2-a3d7-b13af35e94b7 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:37:50.764143 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a331b3e5-85cf-49a2-a3d7-b13af35e94b7 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:37:50.764143 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a331b3e5-85cf-49a2-a3d7-b13af35e94b7 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:37:50.765568 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 723/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 28 17:37:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:37:50.770782 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d82d6e-95f8-4349-8441-657af5f16804 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:37:50.771208 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d82d6e-95f8-4349-8441-657af5f16804 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:37:50.771339 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d82d6e-95f8-4349-8441-657af5f16804 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:37:50.791496 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:37:50.851460 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-c0d82d6e-95f8-4349-8441-657af5f16804 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:37:50.852680 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-c0d82d6e-95f8-4349-8441-657af5f16804 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:37:50.950281 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 701/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 28 17:37:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:37:51.104008 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a331b3e5-85cf-49a2-a3d7-b13af35e94b7 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:37:51.105598 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-a331b3e5-85cf-49a2-a3d7-b13af35e94b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:37:51.219315 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:37:51.257887 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a331b3e5-85cf-49a2-a3d7-b13af35e94b7 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:37:51.375383 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb2734c-fced-477c-ae4e-3aa12f233c34 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd10b49b29df94404a71f43273839c252', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:37:51.375913 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb2734c-fced-477c-ae4e-3aa12f233c34 service nova] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:37:51.376229 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb2734c-fced-477c-ae4e-3aa12f233c34 service nova] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:37:51.376573 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb2734c-fced-477c-ae4e-3aa12f233c34 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:37:51.378289 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-0cb2734c-fced-477c-ae4e-3aa12f233c34 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:37:51.454237 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:37:51.468107 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb2734c-fced-477c-ae4e-3aa12f233c34 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:37:51.470237 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb2734c-fced-477c-ae4e-3aa12f233c34 service nova] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:37:51.489678 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 724/1425] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:37:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:37:52.368136 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5be6e99-4144-46b6-a84e-b5bc9464041f None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:37:52.368705 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5be6e99-4144-46b6-a84e-b5bc9464041f None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:37:52.368705 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5be6e99-4144-46b6-a84e-b5bc9464041f None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:37:52.370095 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 702/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 28 17:37:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 17:37:52.375058 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c377b9cf-34de-4eb3-bb83-ef84a9e89d2e None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:37:52.375466 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c377b9cf-34de-4eb3-bb83-ef84a9e89d2e None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:37:52.375610 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c377b9cf-34de-4eb3-bb83-ef84a9e89d2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:37:52.453069 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-c377b9cf-34de-4eb3-bb83-ef84a9e89d2e None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:37:52.454967 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-c377b9cf-34de-4eb3-bb83-ef84a9e89d2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:37:52.551120 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 725/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 28 17:37:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:37:52.703440 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5be6e99-4144-46b6-a84e-b5bc9464041f None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:37:52.705639 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-f5be6e99-4144-46b6-a84e-b5bc9464041f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:37:52.788873 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:37:52.853013 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5be6e99-4144-46b6-a84e-b5bc9464041f None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:37:52.969204 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4d195e-25ec-410c-9e9c-1d3c0c11a12f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd10b49b29df94404a71f43273839c252', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:37:52.969913 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4d195e-25ec-410c-9e9c-1d3c0c11a12f service nova] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:37:52.970191 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4d195e-25ec-410c-9e9c-1d3c0c11a12f service nova] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:37:52.970465 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4d195e-25ec-410c-9e9c-1d3c0c11a12f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:37:52.972080 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-ba4d195e-25ec-410c-9e9c-1d3c0c11a12f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:37:53.020339 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:37:53.061866 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4d195e-25ec-410c-9e9c-1d3c0c11a12f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:37:53.064106 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4d195e-25ec-410c-9e9c-1d3c0c11a12f service nova] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:37:53.081931 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 703/1428] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:37:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:37:53.896018 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa5d038-6f3e-4bb8-9f28-3e634e6cd12a None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:37:53.896552 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa5d038-6f3e-4bb8-9f28-3e634e6cd12a None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:37:53.896552 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa5d038-6f3e-4bb8-9f28-3e634e6cd12a None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:37:53.897934 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 726/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 28 17:37:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:37:53.902907 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b30cce7-c909-4d34-955a-1be94bc191cf None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:37:53.903355 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b30cce7-c909-4d34-955a-1be94bc191cf None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:37:53.903454 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b30cce7-c909-4d34-955a-1be94bc191cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:37:53.979856 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-2b30cce7-c909-4d34-955a-1be94bc191cf None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:37:53.981078 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-2b30cce7-c909-4d34-955a-1be94bc191cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:37:54.080432 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 704/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 28 17:37:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:37:54.241804 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa5d038-6f3e-4bb8-9f28-3e634e6cd12a None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:37:54.243398 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-2aa5d038-6f3e-4bb8-9f28-3e634e6cd12a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:37:54.292953 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:37:54.396458 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa5d038-6f3e-4bb8-9f28-3e634e6cd12a None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:37:54.508885 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cf7336-8394-4b71-9b2f-d0afdb244979 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd10b49b29df94404a71f43273839c252', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:37:54.509529 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cf7336-8394-4b71-9b2f-d0afdb244979 service nova] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:37:54.509812 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cf7336-8394-4b71-9b2f-d0afdb244979 service nova] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:37:54.510086 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cf7336-8394-4b71-9b2f-d0afdb244979 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:37:54.511720 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-30cf7336-8394-4b71-9b2f-d0afdb244979 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:37:54.518558 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:37:54.608764 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cf7336-8394-4b71-9b2f-d0afdb244979 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:37:54.611501 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cf7336-8394-4b71-9b2f-d0afdb244979 service nova] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:37:54.630356 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 727/1431] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:37:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 17:37:55.438795 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb2cffc-4b94-43da-919d-784953d6d25c None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:37:55.439390 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb2cffc-4b94-43da-919d-784953d6d25c None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:37:55.439390 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb2cffc-4b94-43da-919d-784953d6d25c None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:37:55.440702 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 705/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 28 17:37:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:37:55.445803 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2163350-8f8d-43f4-8c52-25b1903a5194 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:37:55.446230 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2163350-8f8d-43f4-8c52-25b1903a5194 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:37:55.446358 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2163350-8f8d-43f4-8c52-25b1903a5194 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:37:55.522946 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-f2163350-8f8d-43f4-8c52-25b1903a5194 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:37:55.524152 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-f2163350-8f8d-43f4-8c52-25b1903a5194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:37:55.592191 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:37:55.621509 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 728/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 28 17:37:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:37:55.773571 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb2cffc-4b94-43da-919d-784953d6d25c None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:37:55.775038 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-dbb2cffc-4b94-43da-919d-784953d6d25c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:37:55.792547 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:37:55.926136 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb2cffc-4b94-43da-919d-784953d6d25c None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:37:56.017576 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:37:56.043840 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4eaebe-bdca-4960-bf19-032f41461868 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd10b49b29df94404a71f43273839c252', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:37:56.044380 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4eaebe-bdca-4960-bf19-032f41461868 service nova] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:37:56.044651 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4eaebe-bdca-4960-bf19-032f41461868 service nova] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:37:56.044920 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4eaebe-bdca-4960-bf19-032f41461868 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:37:56.046564 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-ad4eaebe-bdca-4960-bf19-032f41461868 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:37:56.136512 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4eaebe-bdca-4960-bf19-032f41461868 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:37:56.138127 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4eaebe-bdca-4960-bf19-032f41461868 service nova] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:37:56.156591 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 706/1434] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:37:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:37:56.966335 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d688f6d-9975-4cd8-ac92-90c199e1b485 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:37:56.966915 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d688f6d-9975-4cd8-ac92-90c199e1b485 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:37:56.966915 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d688f6d-9975-4cd8-ac92-90c199e1b485 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:37:56.968236 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 729/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 28 17:37:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:37:56.973740 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aa423d-50ca-4db8-8fb3-ed343052a453 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:37:56.974188 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aa423d-50ca-4db8-8fb3-ed343052a453 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:37:56.974378 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aa423d-50ca-4db8-8fb3-ed343052a453 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:37:57.051191 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-66aa423d-50ca-4db8-8fb3-ed343052a453 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:37:57.052401 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-66aa423d-50ca-4db8-8fb3-ed343052a453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:37:57.095014 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:37:57.151720 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 707/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 28 17:37:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:37:57.302799 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d688f6d-9975-4cd8-ac92-90c199e1b485 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:37:57.304324 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-2d688f6d-9975-4cd8-ac92-90c199e1b485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:37:57.463091 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d688f6d-9975-4cd8-ac92-90c199e1b485 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:37:57.521556 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:37:57.581931 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e396361-e9e1-423c-9614-c1ae6b8fc44e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd10b49b29df94404a71f43273839c252', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:37:57.582553 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e396361-e9e1-423c-9614-c1ae6b8fc44e service nova] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:37:57.582835 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e396361-e9e1-423c-9614-c1ae6b8fc44e service nova] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:37:57.583112 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e396361-e9e1-423c-9614-c1ae6b8fc44e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:37:57.584751 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-5e396361-e9e1-423c-9614-c1ae6b8fc44e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:37:57.679308 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e396361-e9e1-423c-9614-c1ae6b8fc44e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:37:57.680851 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e396361-e9e1-423c-9614-c1ae6b8fc44e service nova] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:37:57.700909 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 730/1437] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:37:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:37:58.517142 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869f9f86-7ea0-464f-80ce-5b349867acdc None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:37:58.517740 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869f9f86-7ea0-464f-80ce-5b349867acdc None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:37:58.517740 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869f9f86-7ea0-464f-80ce-5b349867acdc None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:37:58.519142 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 708/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 28 17:37:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:37:58.524158 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c191f9-fc91-454f-90a9-bfdb86548957 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:37:58.524592 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c191f9-fc91-454f-90a9-bfdb86548957 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:37:58.524735 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c191f9-fc91-454f-90a9-bfdb86548957 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:37:58.601306 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-25c191f9-fc91-454f-90a9-bfdb86548957 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:37:58.602521 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-25c191f9-fc91-454f-90a9-bfdb86548957 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:37:58.604500 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:37:58.699995 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 731/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 28 17:37:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:37:58.851241 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869f9f86-7ea0-464f-80ce-5b349867acdc None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:37:58.852757 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-869f9f86-7ea0-464f-80ce-5b349867acdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:37:58.998072 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869f9f86-7ea0-464f-80ce-5b349867acdc None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:37:59.028105 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:37:59.120323 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac54f994-a493-4cb2-bcbb-b5b65ea284c3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd10b49b29df94404a71f43273839c252', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:37:59.120874 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac54f994-a493-4cb2-bcbb-b5b65ea284c3 service nova] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:37:59.121169 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac54f994-a493-4cb2-bcbb-b5b65ea284c3 service nova] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:37:59.121460 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac54f994-a493-4cb2-bcbb-b5b65ea284c3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:37:59.123131 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-ac54f994-a493-4cb2-bcbb-b5b65ea284c3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:37:59.213995 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac54f994-a493-4cb2-bcbb-b5b65ea284c3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:37:59.215586 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac54f994-a493-4cb2-bcbb-b5b65ea284c3 service nova] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:37:59.233510 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 709/1440] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:37:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:38:01.149149 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ddb15f-490f-4c5d-954a-090f15234ad6 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:38:01.149700 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ddb15f-490f-4c5d-954a-090f15234ad6 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:38:01.149700 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ddb15f-490f-4c5d-954a-090f15234ad6 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:38:01.150983 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 732/1441] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 17:38:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:38:01.156342 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dcdaf6-9ace-4a8d-a82f-a8b17693a3b3 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:38:01.157250 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dcdaf6-9ace-4a8d-a82f-a8b17693a3b3 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:38:01.157250 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dcdaf6-9ace-4a8d-a82f-a8b17693a3b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:38:01.198936 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:38:01.224439 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-28dcdaf6-9ace-4a8d-a82f-a8b17693a3b3 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:38:01.225677 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-28dcdaf6-9ace-4a8d-a82f-a8b17693a3b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:38:01.319970 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 710/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 17:38:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:38:01.362982 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ddb15f-490f-4c5d-954a-090f15234ad6 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:38:01.364581 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-59ddb15f-490f-4c5d-954a-090f15234ad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:38:01.470255 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ddb15f-490f-4c5d-954a-090f15234ad6 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:38:01.503242 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:38:01.545042 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00f259c-49ea-43dd-ae01-d2486d61fece service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76817cd541294331b0023069092e8e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:38:01.545708 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00f259c-49ea-43dd-ae01-d2486d61fece service neutron] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:38:01.545986 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00f259c-49ea-43dd-ae01-d2486d61fece service neutron] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:38:01.546255 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00f259c-49ea-43dd-ae01-d2486d61fece service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:38:01.548669 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-f00f259c-49ea-43dd-ae01-d2486d61fece service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:38:01.658783 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00f259c-49ea-43dd-ae01-d2486d61fece service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:38:01.660229 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00f259c-49ea-43dd-ae01-d2486d61fece service neutron] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:38:01.680438 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 733/1443] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:38:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:38:03.839585 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae9e7ac-47b1-4eea-88c4-22f8fbe8f43d None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:38:03.840044 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae9e7ac-47b1-4eea-88c4-22f8fbe8f43d None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:38:03.840152 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae9e7ac-47b1-4eea-88c4-22f8fbe8f43d None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:38:03.841542 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 711/1444] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 17:38:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:38:03.847377 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60f923b-1389-4c60-bbd9-0ad14a29b602 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:38:03.847808 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60f923b-1389-4c60-bbd9-0ad14a29b602 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:38:03.847922 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60f923b-1389-4c60-bbd9-0ad14a29b602 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:38:03.914727 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-a60f923b-1389-4c60-bbd9-0ad14a29b602 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:38:03.915958 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-a60f923b-1389-4c60-bbd9-0ad14a29b602 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:38:03.928774 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:38:04.012358 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 734/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 17:38:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:38:04.050801 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae9e7ac-47b1-4eea-88c4-22f8fbe8f43d None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:38:04.052342 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-7ae9e7ac-47b1-4eea-88c4-22f8fbe8f43d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:38:04.155503 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae9e7ac-47b1-4eea-88c4-22f8fbe8f43d None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:38:04.167375 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:38:04.227676 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383c2643-0d13-4479-ac6d-82a5c70fd688 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76817cd541294331b0023069092e8e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:38:04.228293 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383c2643-0d13-4479-ac6d-82a5c70fd688 service neutron] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:38:04.228569 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383c2643-0d13-4479-ac6d-82a5c70fd688 service neutron] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:38:04.228839 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383c2643-0d13-4479-ac6d-82a5c70fd688 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:38:04.230516 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-383c2643-0d13-4479-ac6d-82a5c70fd688 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:38:04.346159 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383c2643-0d13-4479-ac6d-82a5c70fd688 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:38:04.347595 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383c2643-0d13-4479-ac6d-82a5c70fd688 service neutron] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:38:04.365552 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 712/1446] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:38:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:38:06.717621 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58b2187-fc44-43dd-badb-ccbc9d512a5f None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:38:06.718152 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58b2187-fc44-43dd-badb-ccbc9d512a5f None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:38:06.718152 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58b2187-fc44-43dd-badb-ccbc9d512a5f None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:38:06.720303 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 735/1447] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 17:38:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:38:06.725649 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cd722e-f09e-496f-b2b8-2f857d44559a None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:38:06.726078 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cd722e-f09e-496f-b2b8-2f857d44559a None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:38:06.726189 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cd722e-f09e-496f-b2b8-2f857d44559a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:38:06.773663 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:38:06.792535 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-45cd722e-f09e-496f-b2b8-2f857d44559a None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:38:06.793795 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-45cd722e-f09e-496f-b2b8-2f857d44559a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:38:06.904422 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 713/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 17:38:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:38:06.944083 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58b2187-fc44-43dd-badb-ccbc9d512a5f None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:38:06.945690 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-c58b2187-fc44-43dd-badb-ccbc9d512a5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:38:07.051480 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58b2187-fc44-43dd-badb-ccbc9d512a5f None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:38:07.092035 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:38:07.127767 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86afbfd1-cfe7-4ed2-95ad-edb055edce4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76817cd541294331b0023069092e8e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:38:07.128384 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86afbfd1-cfe7-4ed2-95ad-edb055edce4e service neutron] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:38:07.128668 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86afbfd1-cfe7-4ed2-95ad-edb055edce4e service neutron] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:38:07.128944 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86afbfd1-cfe7-4ed2-95ad-edb055edce4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:38:07.130630 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-86afbfd1-cfe7-4ed2-95ad-edb055edce4e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:38:07.247323 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86afbfd1-cfe7-4ed2-95ad-edb055edce4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:38:07.248752 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86afbfd1-cfe7-4ed2-95ad-edb055edce4e service neutron] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:38:07.267667 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 736/1449] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:38:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:38:09.408677 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a24dd28-1092-445d-bf3c-24ea113d7779 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:38:09.409279 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a24dd28-1092-445d-bf3c-24ea113d7779 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:38:09.409279 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a24dd28-1092-445d-bf3c-24ea113d7779 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:38:09.410624 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 714/1450] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 17:38:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:38:09.416107 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca54782c-64b9-45a4-84c7-efc3d142b90f None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:38:09.416535 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca54782c-64b9-45a4-84c7-efc3d142b90f None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:38:09.416644 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca54782c-64b9-45a4-84c7-efc3d142b90f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:38:09.486095 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-ca54782c-64b9-45a4-84c7-efc3d142b90f None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:38:09.487323 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-ca54782c-64b9-45a4-84c7-efc3d142b90f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:38:09.507545 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:38:09.583636 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 737/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 17:38:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:38:09.623427 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a24dd28-1092-445d-bf3c-24ea113d7779 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:38:09.625752 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-7a24dd28-1092-445d-bf3c-24ea113d7779 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:38:09.730602 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a24dd28-1092-445d-bf3c-24ea113d7779 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:38:09.750711 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:38:09.808244 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd2f1ae-0ff5-448b-9c3c-49adb884e7b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76817cd541294331b0023069092e8e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:38:09.808874 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd2f1ae-0ff5-448b-9c3c-49adb884e7b4 service neutron] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:38:09.809173 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd2f1ae-0ff5-448b-9c3c-49adb884e7b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:38:09.809552 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd2f1ae-0ff5-448b-9c3c-49adb884e7b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:38:09.811273 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-ffd2f1ae-0ff5-448b-9c3c-49adb884e7b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:38:09.927276 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd2f1ae-0ff5-448b-9c3c-49adb884e7b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:38:09.928708 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd2f1ae-0ff5-448b-9c3c-49adb884e7b4 service neutron] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:38:09.947137 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 715/1452] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:38:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:38:12.187934 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73df0e9b-a62f-4e8c-a8ab-55f5aeaadb94 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:38:12.188573 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73df0e9b-a62f-4e8c-a8ab-55f5aeaadb94 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:38:12.188766 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73df0e9b-a62f-4e8c-a8ab-55f5aeaadb94 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:38:12.190689 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 738/1453] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 17:38:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:38:12.199837 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dcdef0-bb97-4ab2-ac0f-86651f079ea7 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:38:12.200319 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dcdef0-bb97-4ab2-ac0f-86651f079ea7 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:38:12.200672 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dcdef0-bb97-4ab2-ac0f-86651f079ea7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:38:12.280971 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:38:12.284923 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-70dcdef0-bb97-4ab2-ac0f-86651f079ea7 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:38:12.286352 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-70dcdef0-bb97-4ab2-ac0f-86651f079ea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:38:12.387945 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 716/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 17:38:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:38:12.426922 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73df0e9b-a62f-4e8c-a8ab-55f5aeaadb94 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:38:12.428466 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-73df0e9b-a62f-4e8c-a8ab-55f5aeaadb94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:38:12.542904 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73df0e9b-a62f-4e8c-a8ab-55f5aeaadb94 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:38:12.592501 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:38:12.617500 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a3a666-4734-440f-b274-078d92580116 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76817cd541294331b0023069092e8e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:38:12.618112 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a3a666-4734-440f-b274-078d92580116 service neutron] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:38:12.618393 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a3a666-4734-440f-b274-078d92580116 service neutron] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:38:12.618668 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a3a666-4734-440f-b274-078d92580116 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:38:12.620298 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-a1a3a666-4734-440f-b274-078d92580116 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:38:12.728967 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a3a666-4734-440f-b274-078d92580116 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:38:12.730421 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a3a666-4734-440f-b274-078d92580116 service neutron] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:38:12.748756 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 739/1455] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:38:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:38:14.836837 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f924648-7f40-4ff0-9770-7feb36932bf4 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:38:14.837283 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f924648-7f40-4ff0-9770-7feb36932bf4 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:38:14.837435 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f924648-7f40-4ff0-9770-7feb36932bf4 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:38:14.838824 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 717/1456] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 17:38:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:38:14.844166 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9057e3eb-68d2-4cb8-ab12-f7291db53bb9 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:38:14.844632 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9057e3eb-68d2-4cb8-ab12-f7291db53bb9 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:38:14.844729 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9057e3eb-68d2-4cb8-ab12-f7291db53bb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:38:14.911235 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-9057e3eb-68d2-4cb8-ab12-f7291db53bb9 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:38:14.912467 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-9057e3eb-68d2-4cb8-ab12-f7291db53bb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:38:14.943252 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:38:15.013643 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 740/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 17:38:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:38:15.052053 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f924648-7f40-4ff0-9770-7feb36932bf4 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:38:15.053667 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-9f924648-7f40-4ff0-9770-7feb36932bf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:38:15.160968 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f924648-7f40-4ff0-9770-7feb36932bf4 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:38:15.190378 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:38:15.236814 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359dc9c2-981d-4279-8608-ddd430ca35ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76817cd541294331b0023069092e8e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:38:15.237467 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359dc9c2-981d-4279-8608-ddd430ca35ff service neutron] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:38:15.237749 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359dc9c2-981d-4279-8608-ddd430ca35ff service neutron] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:38:15.238023 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359dc9c2-981d-4279-8608-ddd430ca35ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:38:15.239657 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-359dc9c2-981d-4279-8608-ddd430ca35ff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:38:15.347087 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359dc9c2-981d-4279-8608-ddd430ca35ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:38:15.348504 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359dc9c2-981d-4279-8608-ddd430ca35ff service neutron] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:38:15.366791 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 718/1458] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:38:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:38:17.418771 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd92737-6612-4979-a0b7-5ad834c5f96a None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:38:17.419539 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd92737-6612-4979-a0b7-5ad834c5f96a None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:38:17.419539 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd92737-6612-4979-a0b7-5ad834c5f96a None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:38:17.421048 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 741/1459] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 17:38:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:38:17.427165 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101bef44-de72-4758-80c8-58b3a9f10307 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:38:17.427624 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101bef44-de72-4758-80c8-58b3a9f10307 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:38:17.427724 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101bef44-de72-4758-80c8-58b3a9f10307 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:38:17.493528 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-101bef44-de72-4758-80c8-58b3a9f10307 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:38:17.494748 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-101bef44-de72-4758-80c8-58b3a9f10307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:38:17.503602 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:38:17.594760 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 719/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 17:38:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:38:17.633110 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd92737-6612-4979-a0b7-5ad834c5f96a None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:38:17.634758 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-cdd92737-6612-4979-a0b7-5ad834c5f96a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:38:17.737447 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd92737-6612-4979-a0b7-5ad834c5f96a None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:38:17.793878 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:38:17.812772 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8671f708-682f-425b-8a5e-33382259bc89 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76817cd541294331b0023069092e8e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:38:17.813414 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8671f708-682f-425b-8a5e-33382259bc89 service neutron] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:38:17.813699 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8671f708-682f-425b-8a5e-33382259bc89 service neutron] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:38:17.813974 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8671f708-682f-425b-8a5e-33382259bc89 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:38:17.815621 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-8671f708-682f-425b-8a5e-33382259bc89 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:38:17.927542 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8671f708-682f-425b-8a5e-33382259bc89 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:38:17.928989 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8671f708-682f-425b-8a5e-33382259bc89 service neutron] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:38:17.947074 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 742/1461] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:38:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:38:19.991813 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac113261-5edb-4a57-b7dc-b83a81c7563c None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:38:19.992366 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac113261-5edb-4a57-b7dc-b83a81c7563c None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:38:19.992366 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac113261-5edb-4a57-b7dc-b83a81c7563c None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:38:19.993824 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 720/1462] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 17:38:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:38:19.999093 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794c4cb5-f7bd-4bf4-8395-adb490284de3 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:38:19.999526 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794c4cb5-f7bd-4bf4-8395-adb490284de3 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:38:19.999673 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794c4cb5-f7bd-4bf4-8395-adb490284de3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:38:20.065573 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-794c4cb5-f7bd-4bf4-8395-adb490284de3 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:38:20.066788 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-794c4cb5-f7bd-4bf4-8395-adb490284de3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:38:20.105360 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:38:20.164095 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 743/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 17:38:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:38:20.201528 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac113261-5edb-4a57-b7dc-b83a81c7563c None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:38:20.203726 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-ac113261-5edb-4a57-b7dc-b83a81c7563c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:38:20.306383 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac113261-5edb-4a57-b7dc-b83a81c7563c None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:38:20.343812 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:38:20.380026 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71038758-dc63-46ee-a430-495b764fa102 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76817cd541294331b0023069092e8e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:38:20.380563 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71038758-dc63-46ee-a430-495b764fa102 service neutron] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:38:20.380838 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71038758-dc63-46ee-a430-495b764fa102 service neutron] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:38:20.381111 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71038758-dc63-46ee-a430-495b764fa102 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:38:20.383027 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-71038758-dc63-46ee-a430-495b764fa102 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:38:20.491126 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71038758-dc63-46ee-a430-495b764fa102 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:38:20.492602 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71038758-dc63-46ee-a430-495b764fa102 service neutron] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:38:20.511268 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 721/1464] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:38:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:38:22.561502 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41232cdc-2816-4fc2-a5cb-2c3d485a11cf None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:38:22.562736 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41232cdc-2816-4fc2-a5cb-2c3d485a11cf None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:38:22.562736 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41232cdc-2816-4fc2-a5cb-2c3d485a11cf None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:38:22.564149 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 744/1465] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 17:38:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:38:22.569398 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad730218-2e45-4ae8-9044-dfecfb971981 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:38:22.569827 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad730218-2e45-4ae8-9044-dfecfb971981 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:38:22.569938 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad730218-2e45-4ae8-9044-dfecfb971981 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:38:22.635982 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-ad730218-2e45-4ae8-9044-dfecfb971981 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:38:22.637239 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-ad730218-2e45-4ae8-9044-dfecfb971981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:38:22.660964 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:38:22.740965 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 722/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 17:38:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:38:22.778027 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41232cdc-2816-4fc2-a5cb-2c3d485a11cf None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:38:22.779578 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-41232cdc-2816-4fc2-a5cb-2c3d485a11cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:38:22.886353 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41232cdc-2816-4fc2-a5cb-2c3d485a11cf None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:38:22.953931 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:38:22.961196 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14cbcfe-2bb1-43b8-99de-0fecba2ea84d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76817cd541294331b0023069092e8e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:38:22.961808 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14cbcfe-2bb1-43b8-99de-0fecba2ea84d service neutron] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:38:22.962089 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14cbcfe-2bb1-43b8-99de-0fecba2ea84d service neutron] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:38:22.962366 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14cbcfe-2bb1-43b8-99de-0fecba2ea84d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:38:22.963987 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-e14cbcfe-2bb1-43b8-99de-0fecba2ea84d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:38:23.075984 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14cbcfe-2bb1-43b8-99de-0fecba2ea84d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:38:23.078087 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14cbcfe-2bb1-43b8-99de-0fecba2ea84d service neutron] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:38:23.096961 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 745/1467] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:38:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:38:25.158424 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d59ced-46a6-4508-b79d-0ecb781d9ae1 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:38:25.159083 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d59ced-46a6-4508-b79d-0ecb781d9ae1 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:38:25.159083 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d59ced-46a6-4508-b79d-0ecb781d9ae1 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:38:25.160338 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 723/1468] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 17:38:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:38:25.165593 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97f98e3-a973-4c6c-8755-c1b7a4e79ad5 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:38:25.166041 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97f98e3-a973-4c6c-8755-c1b7a4e79ad5 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:38:25.166215 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97f98e3-a973-4c6c-8755-c1b7a4e79ad5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:38:25.232534 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-a97f98e3-a973-4c6c-8755-c1b7a4e79ad5 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:38:25.234426 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-a97f98e3-a973-4c6c-8755-c1b7a4e79ad5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:38:25.286382 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:38:25.347677 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 746/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 17:38:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:38:25.387373 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d59ced-46a6-4508-b79d-0ecb781d9ae1 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:38:25.388935 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-32d59ced-46a6-4508-b79d-0ecb781d9ae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:38:25.492732 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d59ced-46a6-4508-b79d-0ecb781d9ae1 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:38:25.539151 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:38:25.568246 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4e7245-f843-40d5-87ec-7cd3107cad63 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76817cd541294331b0023069092e8e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:38:25.568857 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4e7245-f843-40d5-87ec-7cd3107cad63 service neutron] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:38:25.569132 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4e7245-f843-40d5-87ec-7cd3107cad63 service neutron] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:38:25.569434 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4e7245-f843-40d5-87ec-7cd3107cad63 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:38:25.571048 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-4b4e7245-f843-40d5-87ec-7cd3107cad63 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:38:25.690141 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4e7245-f843-40d5-87ec-7cd3107cad63 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:38:25.691885 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4e7245-f843-40d5-87ec-7cd3107cad63 service neutron] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:38:25.710300 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 724/1470] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:38:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:38:35.345886 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=22 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:38:35.707899 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=66 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:38:45.438556 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e55620-4dc4-43ad-b59a-dc44ee5305b6 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:38:45.439128 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e55620-4dc4-43ad-b59a-dc44ee5305b6 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:38:45.439128 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e55620-4dc4-43ad-b59a-dc44ee5305b6 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:38:45.440500 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 747/1471] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:38:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:38:45.445914 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533f7bba-d4e9-4984-a3b6-09449d83540c None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:38:45.446301 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533f7bba-d4e9-4984-a3b6-09449d83540c None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:38:45.446465 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533f7bba-d4e9-4984-a3b6-09449d83540c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:38:45.515707 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-533f7bba-d4e9-4984-a3b6-09449d83540c None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:38:45.516932 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-533f7bba-d4e9-4984-a3b6-09449d83540c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:38:45.615049 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 725/1472] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:38:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:38:45.700769 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b40c0f-ff42-4968-ae03-2a87d73a4211 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:38:45.701101 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b40c0f-ff42-4968-ae03-2a87d73a4211 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:38:45.701409 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b40c0f-ff42-4968-ae03-2a87d73a4211 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:38:45.702735 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 748/1473] 10.222.0.89 () {54 vars in 966 bytes} [Thu Jul 28 17:38:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:38:45.707738 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901a2b15-f3d3-43a2-82dc-7e1f5251247f None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:38:45.708029 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901a2b15-f3d3-43a2-82dc-7e1f5251247f None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:38:45.708304 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901a2b15-f3d3-43a2-82dc-7e1f5251247f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:38:45.783078 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-901a2b15-f3d3-43a2-82dc-7e1f5251247f None None] MFA Rules not processed for user `1564db0771224a05b47d8e3cf778c147`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:38:45.784291 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-901a2b15-f3d3-43a2-82dc-7e1f5251247f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:38:45.805089 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:38:45.865066 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 726/1474] 10.222.0.89 () {58 vars in 1063 bytes} [Thu Jul 28 17:38:45 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:38:45.872753 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b40c0f-ff42-4968-ae03-2a87d73a4211 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:38:45.875072 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-78b40c0f-ff42-4968-ae03-2a87d73a4211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:38:45.979003 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b40c0f-ff42-4968-ae03-2a87d73a4211 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:38:46.054594 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013e3bd3-1bcf-42c1-a841-1075d413a593 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1564db0771224a05b47d8e3cf778c147', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:38:46.055213 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013e3bd3-1bcf-42c1-a841-1075d413a593 service glance] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:38:46.055492 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013e3bd3-1bcf-42c1-a841-1075d413a593 service glance] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:38:46.055765 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013e3bd3-1bcf-42c1-a841-1075d413a593 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:38:46.057419 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-013e3bd3-1bcf-42c1-a841-1075d413a593 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:38:46.232361 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:38:46.288347 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013e3bd3-1bcf-42c1-a841-1075d413a593 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:38:46.289869 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013e3bd3-1bcf-42c1-a841-1075d413a593 service glance] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:38:46.307691 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 749/1475] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:38:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:38:48.414417 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a6fae7-3e27-4efd-8d9a-a5cd20961fd5 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:38:48.415007 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a6fae7-3e27-4efd-8d9a-a5cd20961fd5 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:38:48.415007 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a6fae7-3e27-4efd-8d9a-a5cd20961fd5 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:38:48.416335 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 727/1476] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:38:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:38:48.422470 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3716b06f-3cc2-46af-b71f-dae5c03e333a None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:38:48.422925 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3716b06f-3cc2-46af-b71f-dae5c03e333a None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:38:48.423032 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3716b06f-3cc2-46af-b71f-dae5c03e333a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:38:48.488364 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-3716b06f-3cc2-46af-b71f-dae5c03e333a None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:38:48.489612 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-3716b06f-3cc2-46af-b71f-dae5c03e333a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:38:48.586595 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 750/1477] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:38:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:38:48.610004 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a6fae7-3e27-4efd-8d9a-a5cd20961fd5 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:38:48.611542 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-59a6fae7-3e27-4efd-8d9a-a5cd20961fd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:38:48.737737 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a6fae7-3e27-4efd-8d9a-a5cd20961fd5 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:38:48.786598 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:38:48.832809 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd76c2c4-d1a4-49d8-ac87-5f1fa5bb007b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:38:48.833458 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd76c2c4-d1a4-49d8-ac87-5f1fa5bb007b admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:38:48.833739 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd76c2c4-d1a4-49d8-ac87-5f1fa5bb007b admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:38:48.834012 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd76c2c4-d1a4-49d8-ac87-5f1fa5bb007b admin admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:38:48.834711 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd76c2c4-d1a4-49d8-ac87-5f1fa5bb007b admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:38:48.836440 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: WARNING py.warnings [None req-cd76c2c4-d1a4-49d8-ac87-5f1fa5bb007b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 28 17:38:48.836440 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: warnings.warn(msg) Jul 28 17:38:48.836440 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: Jul 28 17:38:48.837626 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd76c2c4-d1a4-49d8-ac87-5f1fa5bb007b admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:38:48.856258 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 728/1478] 10.222.0.89 () {56 vars in 1108 bytes} [Thu Jul 28 17:38:48 2022] GET /identity/v3/projects => generated 2553 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:38:50.887931 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb01211-676f-47e7-bfbd-2b4ea0dafa5c None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:38:50.888957 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb01211-676f-47e7-bfbd-2b4ea0dafa5c None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:38:50.889073 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb01211-676f-47e7-bfbd-2b4ea0dafa5c None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:38:50.890504 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 751/1479] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:38:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:38:50.895765 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cca47e-e40b-4b79-94ef-f26e71feb1b2 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:38:50.896173 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cca47e-e40b-4b79-94ef-f26e71feb1b2 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:38:50.896305 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cca47e-e40b-4b79-94ef-f26e71feb1b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:38:50.964431 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-f8cca47e-e40b-4b79-94ef-f26e71feb1b2 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:38:50.965678 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-f8cca47e-e40b-4b79-94ef-f26e71feb1b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:38:51.061902 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 729/1480] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:38:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:38:51.105310 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb01211-676f-47e7-bfbd-2b4ea0dafa5c None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:38:51.106834 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-0fb01211-676f-47e7-bfbd-2b4ea0dafa5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:38:51.116782 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:38:51.253695 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb01211-676f-47e7-bfbd-2b4ea0dafa5c None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:38:51.335433 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:38:51.368119 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cf5468-fe88-400f-860c-273d159965b7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd10b49b29df94404a71f43273839c252', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:38:51.368721 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cf5468-fe88-400f-860c-273d159965b7 service nova] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:38:51.368996 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cf5468-fe88-400f-860c-273d159965b7 service nova] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:38:51.369298 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cf5468-fe88-400f-860c-273d159965b7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:38:51.370903 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-a7cf5468-fe88-400f-860c-273d159965b7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:38:51.462391 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cf5468-fe88-400f-860c-273d159965b7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:38:51.463978 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cf5468-fe88-400f-860c-273d159965b7 service nova] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:38:51.482626 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 752/1481] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:38:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:38:53.670801 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cb68a6-9080-4b4e-86c1-8deae810913c None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:38:53.671366 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cb68a6-9080-4b4e-86c1-8deae810913c None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:38:53.671366 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cb68a6-9080-4b4e-86c1-8deae810913c None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:38:53.672711 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 730/1482] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:38:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:38:53.677915 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e498d84-baaf-4d4f-bd5c-7b75982dcaff None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:38:53.678325 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e498d84-baaf-4d4f-bd5c-7b75982dcaff None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:38:53.678461 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e498d84-baaf-4d4f-bd5c-7b75982dcaff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:38:53.745600 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-1e498d84-baaf-4d4f-bd5c-7b75982dcaff None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:38:53.746808 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-1e498d84-baaf-4d4f-bd5c-7b75982dcaff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:38:53.794277 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:38:53.848286 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 753/1483] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:38:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:38:53.885929 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cb68a6-9080-4b4e-86c1-8deae810913c None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:38:53.887473 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-69cb68a6-9080-4b4e-86c1-8deae810913c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:38:53.910544 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:38:53.996276 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cb68a6-9080-4b4e-86c1-8deae810913c None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:38:54.071343 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cde53e1-347c-40fc-96a3-ebae37a7a579 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76817cd541294331b0023069092e8e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:38:54.071822 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cde53e1-347c-40fc-96a3-ebae37a7a579 service neutron] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:38:54.072101 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cde53e1-347c-40fc-96a3-ebae37a7a579 service neutron] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:38:54.072363 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cde53e1-347c-40fc-96a3-ebae37a7a579 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:38:54.074207 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-2cde53e1-347c-40fc-96a3-ebae37a7a579 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:38:54.150732 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:38:54.186601 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cde53e1-347c-40fc-96a3-ebae37a7a579 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:38:54.188051 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cde53e1-347c-40fc-96a3-ebae37a7a579 service neutron] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:38:54.205713 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 731/1484] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:38:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:38:56.265130 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c43b815-e56f-4240-9642-fc7754d44e8b None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:38:56.265756 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c43b815-e56f-4240-9642-fc7754d44e8b None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:38:56.265756 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c43b815-e56f-4240-9642-fc7754d44e8b None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:38:56.267060 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 754/1485] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:38:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:38:56.272143 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eb6774-8ba9-4b69-82c7-1e34dca4dab0 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:38:56.272615 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eb6774-8ba9-4b69-82c7-1e34dca4dab0 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:38:56.272744 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eb6774-8ba9-4b69-82c7-1e34dca4dab0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:38:56.338978 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-38eb6774-8ba9-4b69-82c7-1e34dca4dab0 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:38:56.340207 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-38eb6774-8ba9-4b69-82c7-1e34dca4dab0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:38:56.436172 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 732/1486] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:38:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:38:56.474914 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c43b815-e56f-4240-9642-fc7754d44e8b None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:38:56.476501 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-3c43b815-e56f-4240-9642-fc7754d44e8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:38:56.580767 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c43b815-e56f-4240-9642-fc7754d44e8b None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:38:56.653625 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:38:56.655924 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60390697-1735-45e4-912e-1e94767221db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76817cd541294331b0023069092e8e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:38:56.656542 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60390697-1735-45e4-912e-1e94767221db service neutron] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:38:56.657036 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60390697-1735-45e4-912e-1e94767221db service neutron] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:38:56.657381 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60390697-1735-45e4-912e-1e94767221db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:38:56.659477 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-60390697-1735-45e4-912e-1e94767221db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:38:56.780442 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60390697-1735-45e4-912e-1e94767221db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:38:56.781894 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60390697-1735-45e4-912e-1e94767221db service neutron] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:38:56.799689 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 755/1487] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:38:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:38:58.914405 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299f1889-0d18-4fad-a851-a347219a2b93 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:38:58.914936 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299f1889-0d18-4fad-a851-a347219a2b93 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:38:58.914936 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299f1889-0d18-4fad-a851-a347219a2b93 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:38:58.916237 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 733/1488] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:38:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:38:58.921673 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227481b5-249c-4754-aa41-259a8dbc5ac8 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:38:58.922098 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227481b5-249c-4754-aa41-259a8dbc5ac8 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:38:58.922232 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227481b5-249c-4754-aa41-259a8dbc5ac8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:38:58.988793 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-227481b5-249c-4754-aa41-259a8dbc5ac8 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:38:58.990043 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-227481b5-249c-4754-aa41-259a8dbc5ac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:38:59.043063 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:38:59.087735 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 756/1489] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:38:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:38:59.129488 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299f1889-0d18-4fad-a851-a347219a2b93 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:38:59.131320 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-299f1889-0d18-4fad-a851-a347219a2b93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:38:59.167484 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:38:59.237380 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299f1889-0d18-4fad-a851-a347219a2b93 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:38:59.311537 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3271d9-18f6-4e53-9608-2800299cea4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76817cd541294331b0023069092e8e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:38:59.312153 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3271d9-18f6-4e53-9608-2800299cea4c service neutron] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:38:59.312431 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3271d9-18f6-4e53-9608-2800299cea4c service neutron] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:38:59.312701 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3271d9-18f6-4e53-9608-2800299cea4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:38:59.314373 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-dd3271d9-18f6-4e53-9608-2800299cea4c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:38:59.393839 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:38:59.423762 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3271d9-18f6-4e53-9608-2800299cea4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:38:59.425135 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3271d9-18f6-4e53-9608-2800299cea4c service neutron] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:38:59.442819 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 734/1490] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:38:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:39:01.678318 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290d40fb-0dc4-4134-8f73-bb3568a71f5a None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:39:01.678860 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290d40fb-0dc4-4134-8f73-bb3568a71f5a None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:39:01.678860 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290d40fb-0dc4-4134-8f73-bb3568a71f5a None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:39:01.680220 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 757/1491] 10.222.0.89 () {54 vars in 926 bytes} [Thu Jul 28 17:39:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:39:01.686100 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdae364-c6c4-4add-80d0-4c95592163bd None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:39:01.686584 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdae364-c6c4-4add-80d0-4c95592163bd None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:39:01.686716 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdae364-c6c4-4add-80d0-4c95592163bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:39:01.752114 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-dfdae364-c6c4-4add-80d0-4c95592163bd None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:39:01.753369 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-dfdae364-c6c4-4add-80d0-4c95592163bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:39:01.853584 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 735/1492] 10.222.0.89 () {58 vars in 1023 bytes} [Thu Jul 28 17:39:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:39:01.893526 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290d40fb-0dc4-4134-8f73-bb3568a71f5a None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:39:01.895075 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-290d40fb-0dc4-4134-8f73-bb3568a71f5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:39:01.998614 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290d40fb-0dc4-4134-8f73-bb3568a71f5a None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:39:02.072105 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835e124d-d8ac-46a1-b188-998fb262b17b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76817cd541294331b0023069092e8e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:39:02.072718 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835e124d-d8ac-46a1-b188-998fb262b17b service neutron] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:39:02.072993 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835e124d-d8ac-46a1-b188-998fb262b17b service neutron] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:39:02.073298 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835e124d-d8ac-46a1-b188-998fb262b17b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:39:02.074907 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-835e124d-d8ac-46a1-b188-998fb262b17b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:39:02.084026 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:39:02.183847 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835e124d-d8ac-46a1-b188-998fb262b17b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:39:02.186022 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835e124d-d8ac-46a1-b188-998fb262b17b service neutron] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:39:02.205824 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 758/1493] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:39:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:39:11.851688 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=82 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:39:12.203429 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=50 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:40:37.034282 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf49df10-4c98-4099-8b74-77c72babf331 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:37.034885 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf49df10-4c98-4099-8b74-77c72babf331 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:37.034885 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf49df10-4c98-4099-8b74-77c72babf331 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:37.109494 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-cf49df10-4c98-4099-8b74-77c72babf331 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:40:37.110705 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-cf49df10-4c98-4099-8b74-77c72babf331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:40:37.205206 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 736/1494] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:40:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:40:37.213936 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a301cd0e-1753-4f35-971b-1dd09524f95a None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:37.214775 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a301cd0e-1753-4f35-971b-1dd09524f95a None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:37.215055 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a301cd0e-1753-4f35-971b-1dd09524f95a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:37.302588 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-a301cd0e-1753-4f35-971b-1dd09524f95a None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:40:37.303738 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-a301cd0e-1753-4f35-971b-1dd09524f95a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:40:37.398899 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 759/1495] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:40:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:40:37.404279 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf49df10-4c98-4099-8b74-77c72babf331 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:37.405517 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-cf49df10-4c98-4099-8b74-77c72babf331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:40:37.417800 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf49df10-4c98-4099-8b74-77c72babf331 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:37.478601 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:40:37.512681 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1f1939-137c-4362-bca9-633019015d8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:37.513327 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1f1939-137c-4362-bca9-633019015d8b admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:37.513606 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1f1939-137c-4362-bca9-633019015d8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:37.513875 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1f1939-137c-4362-bca9-633019015d8b admin admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:37.514568 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1f1939-137c-4362-bca9-633019015d8b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:40:37.514870 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1f1939-137c-4362-bca9-633019015d8b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:40:37.515877 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: WARNING py.warnings [None req-bf1f1939-137c-4362-bca9-633019015d8b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 28 17:40:37.515877 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: warnings.warn(msg) Jul 28 17:40:37.515877 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: Jul 28 17:40:37.516771 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1f1939-137c-4362-bca9-633019015d8b admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:40:37.528451 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 737/1496] 10.222.0.89 () {58 vars in 1153 bytes} [Thu Jul 28 17:40:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:40:37.536724 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a301cd0e-1753-4f35-971b-1dd09524f95a None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:37.549304 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a301cd0e-1753-4f35-971b-1dd09524f95a None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:37.613713 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:40:37.647802 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b962e89-b57d-4c3f-995f-21991bb7493c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:37.648413 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b962e89-b57d-4c3f-995f-21991bb7493c admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:37.648694 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b962e89-b57d-4c3f-995f-21991bb7493c admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:37.648970 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b962e89-b57d-4c3f-995f-21991bb7493c admin admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:37.649678 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b962e89-b57d-4c3f-995f-21991bb7493c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:40:37.652027 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: WARNING py.warnings [None req-0b962e89-b57d-4c3f-995f-21991bb7493c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 28 17:40:37.652027 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: warnings.warn(msg) Jul 28 17:40:37.652027 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: Jul 28 17:40:37.653254 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b962e89-b57d-4c3f-995f-21991bb7493c admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:40:37.708558 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 760/1497] 10.222.0.89 () {60 vars in 1153 bytes} [Thu Jul 28 17:40:37 2022] POST /identity/v3/projects => generated 377 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:40:37.713049 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1f1939-137c-4362-bca9-633019015d8b admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:37.715352 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-bf1f1939-137c-4362-bca9-633019015d8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:40:37.747762 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1f1939-137c-4362-bca9-633019015d8b admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:37.842486 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30ccf2a-a450-4418-a8cb-db487666c4ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:37.843100 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30ccf2a-a450-4418-a8cb-db487666c4ab admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:37.843379 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30ccf2a-a450-4418-a8cb-db487666c4ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:37.843649 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30ccf2a-a450-4418-a8cb-db487666c4ab admin admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:37.844312 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30ccf2a-a450-4418-a8cb-db487666c4ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:40:37.845338 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: WARNING py.warnings [None req-b30ccf2a-a450-4418-a8cb-db487666c4ab admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 28 17:40:37.845338 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: warnings.warn(msg) Jul 28 17:40:37.845338 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: Jul 28 17:40:37.846572 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30ccf2a-a450-4418-a8cb-db487666c4ab admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:40:37.888183 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 738/1498] 10.222.0.89 () {60 vars in 1144 bytes} [Thu Jul 28 17:40:37 2022] POST /identity/v3/users => generated 289 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:40:37.892018 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b962e89-b57d-4c3f-995f-21991bb7493c admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:37.893566 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-0b962e89-b57d-4c3f-995f-21991bb7493c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:40:37.955435 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b962e89-b57d-4c3f-995f-21991bb7493c admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:38.039649 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:40:38.050711 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0ea481-e7fa-4364-982f-34d55fe0b9ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:38.051323 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0ea481-e7fa-4364-982f-34d55fe0b9ad admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:38.051601 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0ea481-e7fa-4364-982f-34d55fe0b9ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:38.051870 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0ea481-e7fa-4364-982f-34d55fe0b9ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:38.052559 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0ea481-e7fa-4364-982f-34d55fe0b9ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:40:38.053590 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: WARNING py.warnings [None req-3f0ea481-e7fa-4364-982f-34d55fe0b9ad admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 28 17:40:38.053590 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: warnings.warn(msg) Jul 28 17:40:38.053590 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: Jul 28 17:40:38.054471 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0ea481-e7fa-4364-982f-34d55fe0b9ad admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:40:38.067196 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 761/1499] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:40:37 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:40:38.071355 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30ccf2a-a450-4418-a8cb-db487666c4ab admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:38.074384 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-b30ccf2a-a450-4418-a8cb-db487666c4ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:40:38.085362 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30ccf2a-a450-4418-a8cb-db487666c4ab admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:38.111856 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:40:38.184823 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d40cde0-a736-4c98-942a-109bcdefcd8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:38.185487 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d40cde0-a736-4c98-942a-109bcdefcd8c admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:38.185771 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d40cde0-a736-4c98-942a-109bcdefcd8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:38.186046 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d40cde0-a736-4c98-942a-109bcdefcd8c admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:38.186745 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d40cde0-a736-4c98-942a-109bcdefcd8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:40:38.187754 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: WARNING py.warnings [None req-1d40cde0-a736-4c98-942a-109bcdefcd8c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 28 17:40:38.187754 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: warnings.warn(msg) Jul 28 17:40:38.187754 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: Jul 28 17:40:38.188639 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d40cde0-a736-4c98-942a-109bcdefcd8c admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:40:38.201184 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 739/1500] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:40:38 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:40:38.205412 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0ea481-e7fa-4364-982f-34d55fe0b9ad admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:38.206961 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-3f0ea481-e7fa-4364-982f-34d55fe0b9ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:40:38.240660 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0ea481-e7fa-4364-982f-34d55fe0b9ad admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:38.334462 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223349f5-eab9-40da-8f74-3bd086916a25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:38.335124 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223349f5-eab9-40da-8f74-3bd086916a25 admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:38.335398 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223349f5-eab9-40da-8f74-3bd086916a25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:38.335665 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223349f5-eab9-40da-8f74-3bd086916a25 admin admin] PATH_INFO: `/v3/projects/04ac4aa49ccd41eb84068e24a39ca052/users/764509feb44e4272902f56281926d9df/roles/11f8731756fe4caca13efa61b71b11c5` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:38.358485 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223349f5-eab9-40da-8f74-3bd086916a25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04ac4aa49ccd41eb84068e24a39ca052, user_id=764509feb44e4272902f56281926d9df, role_id=11f8731756fe4caca13efa61b71b11c5)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:40:38.359548 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: WARNING py.warnings [None req-223349f5-eab9-40da-8f74-3bd086916a25 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 28 17:40:38.359548 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: warnings.warn(msg) Jul 28 17:40:38.359548 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: Jul 28 17:40:38.361607 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223349f5-eab9-40da-8f74-3bd086916a25 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:40:38.389209 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 762/1501] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:40:38 2022] PUT /identity/v3/projects/04ac4aa49ccd41eb84068e24a39ca052/users/764509feb44e4272902f56281926d9df/roles/11f8731756fe4caca13efa61b71b11c5 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:40:38.401584 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c23ac0-7222-4d54-9308-7ae89c14106c None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:38.401871 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c23ac0-7222-4d54-9308-7ae89c14106c None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:38.402145 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c23ac0-7222-4d54-9308-7ae89c14106c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:38.460046 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-a2c23ac0-7222-4d54-9308-7ae89c14106c None None] MFA Rules not processed for user `764509feb44e4272902f56281926d9df`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:40:38.461216 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-a2c23ac0-7222-4d54-9308-7ae89c14106c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:40:38.538403 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 740/1502] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:40:38 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:40:38.551964 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69834ac-d437-4036-98e9-618e81d89e55 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:38.552252 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69834ac-d437-4036-98e9-618e81d89e55 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:38.552534 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69834ac-d437-4036-98e9-618e81d89e55 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:38.553931 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 763/1503] 10.222.0.89 () {54 vars in 964 bytes} [Thu Jul 28 17:40:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:40:38.560101 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af61acf-10af-45cf-aed7-aeac579c211a None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:38.560388 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af61acf-10af-45cf-aed7-aeac579c211a None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:38.560661 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af61acf-10af-45cf-aed7-aeac579c211a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:38.592287 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:40:38.638864 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-4af61acf-10af-45cf-aed7-aeac579c211a None None] MFA Rules not processed for user `96d512d0c84240918dae6c4a3e8b7b79`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:40:38.640096 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-4af61acf-10af-45cf-aed7-aeac579c211a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:40:38.723072 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 741/1504] 10.222.0.89 () {58 vars in 1061 bytes} [Thu Jul 28 17:40:38 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:40:38.731998 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69834ac-d437-4036-98e9-618e81d89e55 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:38.732685 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-a69834ac-d437-4036-98e9-618e81d89e55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:40:38.783757 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:40:38.841893 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69834ac-d437-4036-98e9-618e81d89e55 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:38.915285 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e2a1f1-b1a8-43b9-b1a7-d7d137a420eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96d512d0c84240918dae6c4a3e8b7b79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:38.915903 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e2a1f1-b1a8-43b9-b1a7-d7d137a420eb service cinder] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:38.916186 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e2a1f1-b1a8-43b9-b1a7-d7d137a420eb service cinder] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:38.916506 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e2a1f1-b1a8-43b9-b1a7-d7d137a420eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:38.918420 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-51e2a1f1-b1a8-43b9-b1a7-d7d137a420eb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:40:39.016847 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e2a1f1-b1a8-43b9-b1a7-d7d137a420eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:40:39.018225 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e2a1f1-b1a8-43b9-b1a7-d7d137a420eb service cinder] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:40:39.021762 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:40:39.036752 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 764/1505] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:40:38 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:40:39.082888 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b116ffe1-1fe9-4693-b893-fc75f1190e34 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:39.083292 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b116ffe1-1fe9-4693-b893-fc75f1190e34 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:39.083451 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b116ffe1-1fe9-4693-b893-fc75f1190e34 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:39.084728 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 742/1506] 10.222.0.89 () {52 vars in 830 bytes} [Thu Jul 28 17:40:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:40:39.088229 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e2a1f1-b1a8-43b9-b1a7-d7d137a420eb service cinder] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:39.090019 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-51e2a1f1-b1a8-43b9-b1a7-d7d137a420eb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:40:39.191866 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e2a1f1-b1a8-43b9-b1a7-d7d137a420eb service cinder] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:39.284723 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe27829-a1c7-4cc2-800b-bc79740fc0f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:39.285413 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe27829-a1c7-4cc2-800b-bc79740fc0f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:39.285567 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe27829-a1c7-4cc2-800b-bc79740fc0f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:39.285835 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe27829-a1c7-4cc2-800b-bc79740fc0f2 admin admin] PATH_INFO: `/v3/users/764509feb44e4272902f56281926d9df` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:39.298740 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe27829-a1c7-4cc2-800b-bc79740fc0f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=764509feb44e4272902f56281926d9df)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:40:39.299754 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: WARNING py.warnings [None req-8fe27829-a1c7-4cc2-800b-bc79740fc0f2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 28 17:40:39.299754 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: warnings.warn(msg) Jul 28 17:40:39.299754 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: Jul 28 17:40:39.300957 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe27829-a1c7-4cc2-800b-bc79740fc0f2 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:40:39.311489 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:40:39.365110 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-8fe27829-a1c7-4cc2-800b-bc79740fc0f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '764509feb44e4272902f56281926d9df'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:40:39.378305 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-8fe27829-a1c7-4cc2-800b-bc79740fc0f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '764509feb44e4272902f56281926d9df'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:40:39.378763 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-8fe27829-a1c7-4cc2-800b-bc79740fc0f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '764509feb44e4272902f56281926d9df'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:40:39.388800 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-8fe27829-a1c7-4cc2-800b-bc79740fc0f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '764509feb44e4272902f56281926d9df'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:40:39.406780 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-8fe27829-a1c7-4cc2-800b-bc79740fc0f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '764509feb44e4272902f56281926d9df'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:40:39.412908 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 765/1507] 10.222.0.89 () {58 vars in 1224 bytes} [Thu Jul 28 17:40:39 2022] DELETE /identity/v3/users/764509feb44e4272902f56281926d9df => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:40:39.417939 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c08802-fab7-46f0-8014-fdf80db3b70d None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:39.418268 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c08802-fab7-46f0-8014-fdf80db3b70d None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:39.418490 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c08802-fab7-46f0-8014-fdf80db3b70d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:39.474383 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-f2c08802-fab7-46f0-8014-fdf80db3b70d None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:40:39.475598 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-f2c08802-fab7-46f0-8014-fdf80db3b70d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:40:39.550816 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:40:39.564609 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 743/1508] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:40:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:40:39.574301 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe27829-a1c7-4cc2-800b-bc79740fc0f2 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:39.575527 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-8fe27829-a1c7-4cc2-800b-bc79740fc0f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:40:39.675065 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe27829-a1c7-4cc2-800b-bc79740fc0f2 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:39.732481 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:40:39.752643 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a9b471-6b01-412a-be54-a13359a4a023 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76817cd541294331b0023069092e8e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:39.753297 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a9b471-6b01-412a-be54-a13359a4a023 service neutron] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:39.753576 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a9b471-6b01-412a-be54-a13359a4a023 service neutron] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:39.753846 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a9b471-6b01-412a-be54-a13359a4a023 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:39.755732 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-74a9b471-6b01-412a-be54-a13359a4a023 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:40:39.865169 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a9b471-6b01-412a-be54-a13359a4a023 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:40:39.866696 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a9b471-6b01-412a-be54-a13359a4a023 service neutron] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:40:39.885942 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 766/1509] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:40:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 17:40:40.216179 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c08802-fab7-46f0-8014-fdf80db3b70d None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:40:40.217456 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-f2c08802-fab7-46f0-8014-fdf80db3b70d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:40:40.228981 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c08802-fab7-46f0-8014-fdf80db3b70d None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:40:40.328263 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d03277-dcd0-4a1d-835f-3f2bb650ac5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:40:40.328897 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d03277-dcd0-4a1d-835f-3f2bb650ac5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:40:40.329196 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d03277-dcd0-4a1d-835f-3f2bb650ac5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:40:40.329473 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d03277-dcd0-4a1d-835f-3f2bb650ac5b admin admin] PATH_INFO: `/v3/projects/04ac4aa49ccd41eb84068e24a39ca052` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:40:40.340080 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d03277-dcd0-4a1d-835f-3f2bb650ac5b admin admin] RBAC: Authorizing `identity:delete_project(project_id=04ac4aa49ccd41eb84068e24a39ca052)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:40:40.341095 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: WARNING py.warnings [None req-11d03277-dcd0-4a1d-835f-3f2bb650ac5b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 28 17:40:40.341095 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: warnings.warn(msg) Jul 28 17:40:40.341095 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: Jul 28 17:40:40.342371 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d03277-dcd0-4a1d-835f-3f2bb650ac5b admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:40:40.399191 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-11d03277-dcd0-4a1d-835f-3f2bb650ac5b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04ac4aa49ccd41eb84068e24a39ca052', 'request_id': 'req-11d03277-dcd0-4a1d-835f-3f2bb650ac5b'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:40:40.414147 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-11d03277-dcd0-4a1d-835f-3f2bb650ac5b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04ac4aa49ccd41eb84068e24a39ca052', 'request_id': 'req-11d03277-dcd0-4a1d-835f-3f2bb650ac5b'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:40:40.421420 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-11d03277-dcd0-4a1d-835f-3f2bb650ac5b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04ac4aa49ccd41eb84068e24a39ca052', 'request_id': 'req-11d03277-dcd0-4a1d-835f-3f2bb650ac5b'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:40:40.447240 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-11d03277-dcd0-4a1d-835f-3f2bb650ac5b admin admin] The token cache is being invalidate because project 04ac4aa49ccd41eb84068e24a39ca052 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80325) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 17:40:40.447532 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-11d03277-dcd0-4a1d-835f-3f2bb650ac5b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:40:40.448165 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-11d03277-dcd0-4a1d-835f-3f2bb650ac5b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:40:40.449642 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 744/1510] 10.222.0.89 () {58 vars in 1233 bytes} [Thu Jul 28 17:40:40 2022] DELETE /identity/v3/projects/04ac4aa49ccd41eb84068e24a39ca052 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:40:49.883048 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=62 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:40:50.445131 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:23.363202 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282c765d-09fc-4693-9956-a3ef07b50859 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:23.363813 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282c765d-09fc-4693-9956-a3ef07b50859 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:23.363813 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282c765d-09fc-4693-9956-a3ef07b50859 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:23.365141 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 767/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 28 17:42:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:42:23.370093 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69152aba-6aa4-4c91-8ab5-aefcaeffde29 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:23.370526 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69152aba-6aa4-4c91-8ab5-aefcaeffde29 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:23.370643 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69152aba-6aa4-4c91-8ab5-aefcaeffde29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:23.451439 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-69152aba-6aa4-4c91-8ab5-aefcaeffde29 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:42:23.453360 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-69152aba-6aa4-4c91-8ab5-aefcaeffde29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:42:23.558797 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 745/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 28 17:42:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 17:42:23.716996 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282c765d-09fc-4693-9956-a3ef07b50859 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:23.719759 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-282c765d-09fc-4693-9956-a3ef07b50859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:42:23.841036 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282c765d-09fc-4693-9956-a3ef07b50859 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:23.847222 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4087b3b-3e16-4578-83ab-bc306b3662da service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd10b49b29df94404a71f43273839c252', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:23.847840 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4087b3b-3e16-4578-83ab-bc306b3662da service nova] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:23.848121 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4087b3b-3e16-4578-83ab-bc306b3662da service nova] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:23.848398 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4087b3b-3e16-4578-83ab-bc306b3662da service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:23.856890 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4087b3b-3e16-4578-83ab-bc306b3662da service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:23.859334 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4087b3b-3e16-4578-83ab-bc306b3662da service nova] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:23.868783 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 768/1513] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:42:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:42:24.704986 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7efbc51-277f-421c-b0b0-644f51c6ac06 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:24.705584 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7efbc51-277f-421c-b0b0-644f51c6ac06 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:24.705692 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7efbc51-277f-421c-b0b0-644f51c6ac06 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:24.707020 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 746/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 28 17:42:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:42:24.711738 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2ed36a-d9d4-4e40-95aa-82db28577a97 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:24.712167 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2ed36a-d9d4-4e40-95aa-82db28577a97 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:24.712307 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2ed36a-d9d4-4e40-95aa-82db28577a97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:24.732208 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-2e2ed36a-d9d4-4e40-95aa-82db28577a97 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:42:24.733481 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-2e2ed36a-d9d4-4e40-95aa-82db28577a97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:42:24.741596 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 769/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 28 17:42:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:42:24.892955 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7efbc51-277f-421c-b0b0-644f51c6ac06 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:24.907817 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7efbc51-277f-421c-b0b0-644f51c6ac06 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:24.913337 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997e0c1c-6468-4a01-bed7-edafa31973de service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd10b49b29df94404a71f43273839c252', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:24.913942 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997e0c1c-6468-4a01-bed7-edafa31973de service nova] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:24.914226 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997e0c1c-6468-4a01-bed7-edafa31973de service nova] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:24.914504 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997e0c1c-6468-4a01-bed7-edafa31973de service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:24.922683 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997e0c1c-6468-4a01-bed7-edafa31973de service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:24.924263 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997e0c1c-6468-4a01-bed7-edafa31973de service nova] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:24.933896 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 747/1516] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:42:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:42:26.270677 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44460ffc-5c0e-4433-ac65-e705ce91e65d None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:26.271280 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44460ffc-5c0e-4433-ac65-e705ce91e65d None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:26.271280 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44460ffc-5c0e-4433-ac65-e705ce91e65d None None] PATH_INFO: `/v3` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:26.272483 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 770/1517] 10.222.0.100 () {54 vars in 926 bytes} [Thu Jul 28 17:42:26 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:42:26.281319 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c762b80e-fe06-4e94-932d-84373babdbf6 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:26.281747 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c762b80e-fe06-4e94-932d-84373babdbf6 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:26.281870 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c762b80e-fe06-4e94-932d-84373babdbf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:26.340318 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-c762b80e-fe06-4e94-932d-84373babdbf6 None None] MFA Rules not processed for user `e4bd290018394b0d955cd376ce35b77f`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:42:26.341567 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-c762b80e-fe06-4e94-932d-84373babdbf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:42:26.363990 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:26.419539 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 748/1518] 10.222.0.100 () {58 vars in 1016 bytes} [Thu Jul 28 17:42:26 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:42:34.727462 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=56 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:42:36.175338 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9a8916-c61b-4adb-8e3a-a25dbd78e37c None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:36.175969 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9a8916-c61b-4adb-8e3a-a25dbd78e37c None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:36.175969 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9a8916-c61b-4adb-8e3a-a25dbd78e37c None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:36.177429 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 771/1519] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 28 17:42:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:42:36.182041 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2073547c-072b-4252-bb31-c4a65568f007 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:36.182524 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ec2d96-9327-4aa2-b059-d43d5a026ad6 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:36.182639 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2073547c-072b-4252-bb31-c4a65568f007 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:36.182747 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ec2d96-9327-4aa2-b059-d43d5a026ad6 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:36.182848 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2073547c-072b-4252-bb31-c4a65568f007 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:36.182944 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ec2d96-9327-4aa2-b059-d43d5a026ad6 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:36.184056 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 772/1520] 10.222.0.89 () {54 vars in 961 bytes} [Thu Jul 28 17:42:36 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 17:42:36.189539 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844b8f68-530d-447b-a827-cb2d57633a20 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:36.189827 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844b8f68-530d-447b-a827-cb2d57633a20 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:36.190099 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844b8f68-530d-447b-a827-cb2d57633a20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:36.202870 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-2073547c-072b-4252-bb31-c4a65568f007 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:42:36.204186 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-2073547c-072b-4252-bb31-c4a65568f007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:42:36.209929 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-844b8f68-530d-447b-a827-cb2d57633a20 None None] MFA Rules not processed for user `e4bd290018394b0d955cd376ce35b77f`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:42:36.211118 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-844b8f68-530d-447b-a827-cb2d57633a20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:42:36.212205 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 749/1521] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 28 17:42:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:42:36.220652 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 773/1522] 10.222.0.89 () {58 vars in 1058 bytes} [Thu Jul 28 17:42:36 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 17:42:36.228833 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2073547c-072b-4252-bb31-c4a65568f007 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:36.240561 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2073547c-072b-4252-bb31-c4a65568f007 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:36.245630 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457fe94a-2390-470e-b9ad-3de2a17c3c2b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4bd290018394b0d955cd376ce35b77f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:36.246258 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457fe94a-2390-470e-b9ad-3de2a17c3c2b service placement] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:36.246541 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457fe94a-2390-470e-b9ad-3de2a17c3c2b service placement] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:36.246814 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457fe94a-2390-470e-b9ad-3de2a17c3c2b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:36.255334 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457fe94a-2390-470e-b9ad-3de2a17c3c2b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:36.257326 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457fe94a-2390-470e-b9ad-3de2a17c3c2b service placement] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:36.267958 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 750/1523] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:42:36 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:42:36.366927 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844b8f68-530d-447b-a827-cb2d57633a20 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:36.380139 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844b8f68-530d-447b-a827-cb2d57633a20 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:36.385989 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40b2845-b569-4916-bba7-cef8bb7f7c14 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd10b49b29df94404a71f43273839c252', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:36.386598 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40b2845-b569-4916-bba7-cef8bb7f7c14 service nova] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:36.386883 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40b2845-b569-4916-bba7-cef8bb7f7c14 service nova] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:36.387159 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40b2845-b569-4916-bba7-cef8bb7f7c14 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:36.396729 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40b2845-b569-4916-bba7-cef8bb7f7c14 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:36.398334 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40b2845-b569-4916-bba7-cef8bb7f7c14 service nova] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:36.408400 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 774/1524] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:42:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:42:37.257767 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaf6146-2f66-4737-a70c-ee68baad9812 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:37.258304 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaf6146-2f66-4737-a70c-ee68baad9812 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:37.258304 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaf6146-2f66-4737-a70c-ee68baad9812 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:37.259597 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 751/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 28 17:42:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 17:42:37.265117 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee75a0c-d3a2-4b5d-bd9d-704f4813a933 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:37.265566 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee75a0c-d3a2-4b5d-bd9d-704f4813a933 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:37.265698 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee75a0c-d3a2-4b5d-bd9d-704f4813a933 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:37.285881 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-9ee75a0c-d3a2-4b5d-bd9d-704f4813a933 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:42:37.287090 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-9ee75a0c-d3a2-4b5d-bd9d-704f4813a933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:42:37.295023 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 775/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 28 17:42:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:42:37.450309 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaf6146-2f66-4737-a70c-ee68baad9812 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:37.465283 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaf6146-2f66-4737-a70c-ee68baad9812 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:37.470979 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb1b4d2-4370-4192-b6af-5c5cac29433f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd10b49b29df94404a71f43273839c252', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:37.471576 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb1b4d2-4370-4192-b6af-5c5cac29433f service nova] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:37.471857 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb1b4d2-4370-4192-b6af-5c5cac29433f service nova] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:37.472134 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb1b4d2-4370-4192-b6af-5c5cac29433f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:37.480224 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb1b4d2-4370-4192-b6af-5c5cac29433f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:37.481821 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb1b4d2-4370-4192-b6af-5c5cac29433f service nova] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:37.491630 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 752/1527] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:42:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:42:41.495588 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19feefbc-7db6-4db2-bc7a-464a4cad1b8f None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:41.496181 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19feefbc-7db6-4db2-bc7a-464a4cad1b8f None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:41.496181 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19feefbc-7db6-4db2-bc7a-464a4cad1b8f None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:41.497494 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 776/1528] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 17:42:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:42:41.503482 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c97c77-814f-4ef4-a58b-3b8d22d78eba None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:41.503959 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c97c77-814f-4ef4-a58b-3b8d22d78eba None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:41.504073 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c97c77-814f-4ef4-a58b-3b8d22d78eba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:41.523624 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-d0c97c77-814f-4ef4-a58b-3b8d22d78eba None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:42:41.524845 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-d0c97c77-814f-4ef4-a58b-3b8d22d78eba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:42:41.533081 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 753/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 17:42:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:42:41.606077 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19feefbc-7db6-4db2-bc7a-464a4cad1b8f None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:41.607768 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-19feefbc-7db6-4db2-bc7a-464a4cad1b8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:42:41.687717 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19feefbc-7db6-4db2-bc7a-464a4cad1b8f None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:41.691354 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe7c685-dd76-4f03-b2dd-cc3aa9847090 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76817cd541294331b0023069092e8e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:41.691963 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe7c685-dd76-4f03-b2dd-cc3aa9847090 service neutron] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:41.692239 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe7c685-dd76-4f03-b2dd-cc3aa9847090 service neutron] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:41.692511 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe7c685-dd76-4f03-b2dd-cc3aa9847090 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:41.701634 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe7c685-dd76-4f03-b2dd-cc3aa9847090 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:41.703173 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe7c685-dd76-4f03-b2dd-cc3aa9847090 service neutron] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:41.712515 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 777/1530] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:42:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:42:43.731800 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0c7959-533b-4969-9922-e198663bb329 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:43.732382 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0c7959-533b-4969-9922-e198663bb329 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:43.732382 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0c7959-533b-4969-9922-e198663bb329 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:43.734302 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 754/1531] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 17:42:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:42:43.739385 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f92fed-b999-4e1b-8922-691528a531b0 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:43.739810 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f92fed-b999-4e1b-8922-691528a531b0 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:43.739962 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f92fed-b999-4e1b-8922-691528a531b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:43.759524 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-c0f92fed-b999-4e1b-8922-691528a531b0 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:42:43.760725 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-c0f92fed-b999-4e1b-8922-691528a531b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:42:43.769934 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 778/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 17:42:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:42:43.806253 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0c7959-533b-4969-9922-e198663bb329 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:43.817901 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0c7959-533b-4969-9922-e198663bb329 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:43.821440 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77f3517-a7eb-43c2-a4aa-4e9d2dc65435 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76817cd541294331b0023069092e8e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:43.822038 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77f3517-a7eb-43c2-a4aa-4e9d2dc65435 service neutron] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:43.822314 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77f3517-a7eb-43c2-a4aa-4e9d2dc65435 service neutron] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:43.822585 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77f3517-a7eb-43c2-a4aa-4e9d2dc65435 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:43.833055 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77f3517-a7eb-43c2-a4aa-4e9d2dc65435 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:43.834505 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77f3517-a7eb-43c2-a4aa-4e9d2dc65435 service neutron] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:43.844715 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 755/1533] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:42:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 17:42:47.358556 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6df43e-a326-481d-be33-421f8aa0af4e None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:47.359088 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6df43e-a326-481d-be33-421f8aa0af4e None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:47.359088 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6df43e-a326-481d-be33-421f8aa0af4e None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:47.360366 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 779/1534] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 17:42:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:42:47.365712 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234c997c-2cae-4ea8-b7c8-55c83aca12c4 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:47.366149 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234c997c-2cae-4ea8-b7c8-55c83aca12c4 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:47.366268 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234c997c-2cae-4ea8-b7c8-55c83aca12c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:47.387155 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-234c997c-2cae-4ea8-b7c8-55c83aca12c4 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:42:47.388371 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-234c997c-2cae-4ea8-b7c8-55c83aca12c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:42:47.396919 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 756/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 17:42:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:42:47.709239 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6df43e-a326-481d-be33-421f8aa0af4e None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:47.714231 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-2e6df43e-a326-481d-be33-421f8aa0af4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:42:47.790383 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6df43e-a326-481d-be33-421f8aa0af4e None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:47.794151 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3e331b-d440-4f10-b112-9f681de591b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96d512d0c84240918dae6c4a3e8b7b79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:47.794766 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3e331b-d440-4f10-b112-9f681de591b1 service cinder] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:47.795051 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3e331b-d440-4f10-b112-9f681de591b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:47.795363 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3e331b-d440-4f10-b112-9f681de591b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:47.804513 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3e331b-d440-4f10-b112-9f681de591b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:47.805976 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3e331b-d440-4f10-b112-9f681de591b1 service cinder] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:47.816206 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 780/1536] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:42:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:42:49.839483 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f926bc91-4d3e-4338-bf0b-d57ffac66b06 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:49.839960 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f926bc91-4d3e-4338-bf0b-d57ffac66b06 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:49.840066 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f926bc91-4d3e-4338-bf0b-d57ffac66b06 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:49.841434 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 757/1537] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 17:42:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:42:49.846512 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aea214e-12f4-499f-82cc-940d6712f12c None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:49.846973 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aea214e-12f4-499f-82cc-940d6712f12c None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:49.847056 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aea214e-12f4-499f-82cc-940d6712f12c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:49.867842 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-1aea214e-12f4-499f-82cc-940d6712f12c None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:42:49.869055 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-1aea214e-12f4-499f-82cc-940d6712f12c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:42:49.877875 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 781/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 17:42:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:42:50.184865 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f926bc91-4d3e-4338-bf0b-d57ffac66b06 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:42:50.186863 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-f926bc91-4d3e-4338-bf0b-d57ffac66b06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:42:50.199893 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f926bc91-4d3e-4338-bf0b-d57ffac66b06 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:42:50.204221 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f74fb65-d506-43a3-90ae-10a560de98cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96d512d0c84240918dae6c4a3e8b7b79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:42:50.204830 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f74fb65-d506-43a3-90ae-10a560de98cd service cinder] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:42:50.205143 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f74fb65-d506-43a3-90ae-10a560de98cd service cinder] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:42:50.205440 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f74fb65-d506-43a3-90ae-10a560de98cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:42:50.214627 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f74fb65-d506-43a3-90ae-10a560de98cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:42:50.216045 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f74fb65-d506-43a3-90ae-10a560de98cd service cinder] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:42:50.226115 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 758/1539] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:42:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:42:59.863032 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=88 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:43:00.222009 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=56 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:44:20.867138 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66e4668-30b8-4ef4-ac63-2cec6503d86c None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:20.867777 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66e4668-30b8-4ef4-ac63-2cec6503d86c None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:20.867777 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66e4668-30b8-4ef4-ac63-2cec6503d86c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:20.882283 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4187f1e7-3bef-48c9-8712-e96113e23a32 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:20.883635 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4187f1e7-3bef-48c9-8712-e96113e23a32 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:20.884141 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4187f1e7-3bef-48c9-8712-e96113e23a32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:20.892526 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-b66e4668-30b8-4ef4-ac63-2cec6503d86c None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:44:20.893795 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-b66e4668-30b8-4ef4-ac63-2cec6503d86c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:20.909699 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 782/1540] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:44:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:44:20.911002 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c902752-bec4-498c-9c55-e2c5556421c8 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:20.911288 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c902752-bec4-498c-9c55-e2c5556421c8 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:20.911563 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c902752-bec4-498c-9c55-e2c5556421c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:20.929319 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-4187f1e7-3bef-48c9-8712-e96113e23a32 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:44:20.930563 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-4187f1e7-3bef-48c9-8712-e96113e23a32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:20.940570 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-1c902752-bec4-498c-9c55-e2c5556421c8 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:44:20.942224 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-1c902752-bec4-498c-9c55-e2c5556421c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:20.943909 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 759/1541] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:44:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 17:44:20.945205 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6c1bb4-db0c-4e37-a1b2-366fd5037f93 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:20.945496 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6c1bb4-db0c-4e37-a1b2-366fd5037f93 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:20.945772 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6c1bb4-db0c-4e37-a1b2-366fd5037f93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:20.963541 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 783/1542] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:44:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:44:20.964908 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25e7432-d9bd-4ee0-b9d3-7805e6461542 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:20.965240 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25e7432-d9bd-4ee0-b9d3-7805e6461542 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:20.965523 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25e7432-d9bd-4ee0-b9d3-7805e6461542 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:20.985635 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-2d6c1bb4-db0c-4e37-a1b2-366fd5037f93 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:44:20.987851 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-2d6c1bb4-db0c-4e37-a1b2-366fd5037f93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:20.997005 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-d25e7432-d9bd-4ee0-b9d3-7805e6461542 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:44:20.998472 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-d25e7432-d9bd-4ee0-b9d3-7805e6461542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:21.006347 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 760/1543] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:44:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:44:21.009143 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9a13a7-08f0-43ca-9c96-780b96218ffc None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:21.014492 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 784/1544] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:44:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:44:21.014492 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357389cd-e978-45d9-8d06-6427014775e0 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:21.014492 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357389cd-e978-45d9-8d06-6427014775e0 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:21.014492 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357389cd-e978-45d9-8d06-6427014775e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:21.015732 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9a13a7-08f0-43ca-9c96-780b96218ffc None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:21.015732 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9a13a7-08f0-43ca-9c96-780b96218ffc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:21.035757 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-3a9a13a7-08f0-43ca-9c96-780b96218ffc None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:44:21.036922 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-3a9a13a7-08f0-43ca-9c96-780b96218ffc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:21.046385 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 761/1545] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:44:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:44:21.047345 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ad7d44-84fc-4431-b431-891f64a019c8 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:21.047490 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-357389cd-e978-45d9-8d06-6427014775e0 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:44:21.047681 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ad7d44-84fc-4431-b431-891f64a019c8 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:21.047903 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ad7d44-84fc-4431-b431-891f64a019c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:21.052761 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-357389cd-e978-45d9-8d06-6427014775e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:21.063649 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 785/1546] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:44:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:44:21.064926 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373453ac-8a5c-463e-a7bd-80bbbc79f311 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:21.065226 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373453ac-8a5c-463e-a7bd-80bbbc79f311 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:21.065500 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373453ac-8a5c-463e-a7bd-80bbbc79f311 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:21.071607 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-40ad7d44-84fc-4431-b431-891f64a019c8 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:44:21.073147 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-40ad7d44-84fc-4431-b431-891f64a019c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:21.082652 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 762/1547] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:44:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:44:21.083313 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ad7d44-84fc-4431-b431-891f64a019c8 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:21.087038 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-373453ac-8a5c-463e-a7bd-80bbbc79f311 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:44:21.088239 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-373453ac-8a5c-463e-a7bd-80bbbc79f311 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:21.092938 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ad7d44-84fc-4431-b431-891f64a019c8 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:21.097486 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 786/1548] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:44:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:44:21.099151 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054e751f-2eee-4d4e-bfb2-ecf2a098efda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:21.099775 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054e751f-2eee-4d4e-bfb2-ecf2a098efda admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:21.100053 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054e751f-2eee-4d4e-bfb2-ecf2a098efda admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:21.100325 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054e751f-2eee-4d4e-bfb2-ecf2a098efda admin admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:21.101029 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054e751f-2eee-4d4e-bfb2-ecf2a098efda admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:44:21.101372 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054e751f-2eee-4d4e-bfb2-ecf2a098efda admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:21.102130 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9c2d1b-3995-4b89-84a2-af1b1899b9ea None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:21.102413 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9c2d1b-3995-4b89-84a2-af1b1899b9ea None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:21.102690 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9c2d1b-3995-4b89-84a2-af1b1899b9ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:21.103013 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054e751f-2eee-4d4e-bfb2-ecf2a098efda admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:21.115565 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 763/1549] 10.222.0.89 () {58 vars in 1153 bytes} [Thu Jul 28 17:44:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:44:21.117494 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8fdc74-f8a2-4926-96a1-c2bc67e49a4b None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:21.117784 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8fdc74-f8a2-4926-96a1-c2bc67e49a4b None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:21.118061 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8fdc74-f8a2-4926-96a1-c2bc67e49a4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:21.125475 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-4f9c2d1b-3995-4b89-84a2-af1b1899b9ea None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:44:21.126699 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-4f9c2d1b-3995-4b89-84a2-af1b1899b9ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:21.134876 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 787/1550] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:44:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:44:21.135659 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9c2d1b-3995-4b89-84a2-af1b1899b9ea None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:21.141106 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-df8fdc74-f8a2-4926-96a1-c2bc67e49a4b None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:44:21.142360 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-df8fdc74-f8a2-4926-96a1-c2bc67e49a4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:21.147321 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9c2d1b-3995-4b89-84a2-af1b1899b9ea None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:21.150704 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 764/1551] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:44:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:44:21.151899 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdb7a7e-1d75-48a7-865b-e54700594ba1 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:21.152183 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdb7a7e-1d75-48a7-865b-e54700594ba1 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:21.152450 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdb7a7e-1d75-48a7-865b-e54700594ba1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:21.154461 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f79b96-dae0-462a-af2d-13ee589b12d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:21.155083 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f79b96-dae0-462a-af2d-13ee589b12d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:21.155364 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f79b96-dae0-462a-af2d-13ee589b12d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:21.156348 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f79b96-dae0-462a-af2d-13ee589b12d0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:21.157050 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f79b96-dae0-462a-af2d-13ee589b12d0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:44:21.157384 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f79b96-dae0-462a-af2d-13ee589b12d0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:21.158399 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: WARNING py.warnings [None req-71f79b96-dae0-462a-af2d-13ee589b12d0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 28 17:44:21.158399 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: warnings.warn(msg) Jul 28 17:44:21.158399 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: Jul 28 17:44:21.159282 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f79b96-dae0-462a-af2d-13ee589b12d0 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:21.171273 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 788/1552] 10.222.0.89 () {58 vars in 1153 bytes} [Thu Jul 28 17:44:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:44:21.172716 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3b2dae-f629-4e27-a80d-b12b8c3d1998 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:21.173000 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3b2dae-f629-4e27-a80d-b12b8c3d1998 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:21.173313 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3b2dae-f629-4e27-a80d-b12b8c3d1998 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:21.179862 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-fcdb7a7e-1d75-48a7-865b-e54700594ba1 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:44:21.181134 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-fcdb7a7e-1d75-48a7-865b-e54700594ba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:21.189463 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 765/1553] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:44:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:44:21.190672 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593fbc0c-e3cb-427e-9e28-bc4ee778361b None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:21.190954 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593fbc0c-e3cb-427e-9e28-bc4ee778361b None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:21.191228 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593fbc0c-e3cb-427e-9e28-bc4ee778361b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:21.198496 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-5a3b2dae-f629-4e27-a80d-b12b8c3d1998 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:44:21.199692 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-5a3b2dae-f629-4e27-a80d-b12b8c3d1998 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:21.208448 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 789/1554] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:44:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:44:21.209378 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3b2dae-f629-4e27-a80d-b12b8c3d1998 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:21.211554 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-593fbc0c-e3cb-427e-9e28-bc4ee778361b None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:44:21.212757 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-593fbc0c-e3cb-427e-9e28-bc4ee778361b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:21.220047 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3b2dae-f629-4e27-a80d-b12b8c3d1998 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:21.222054 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 766/1555] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:44:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:44:21.222898 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593fbc0c-e3cb-427e-9e28-bc4ee778361b None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:21.224670 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcba13a-416c-496d-bbbd-465c576c27b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:21.225308 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcba13a-416c-496d-bbbd-465c576c27b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:21.225910 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcba13a-416c-496d-bbbd-465c576c27b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:21.226189 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcba13a-416c-496d-bbbd-465c576c27b7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:21.226886 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcba13a-416c-496d-bbbd-465c576c27b7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:44:21.227189 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcba13a-416c-496d-bbbd-465c576c27b7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:21.229322 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcba13a-416c-496d-bbbd-465c576c27b7 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:21.232556 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593fbc0c-e3cb-427e-9e28-bc4ee778361b None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:21.237755 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2eacf8-24a1-4b5a-80d0-167b210db967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:21.238372 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2eacf8-24a1-4b5a-80d0-167b210db967 admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:21.238652 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2eacf8-24a1-4b5a-80d0-167b210db967 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:21.238922 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2eacf8-24a1-4b5a-80d0-167b210db967 admin admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:21.239584 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2eacf8-24a1-4b5a-80d0-167b210db967 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:21.240594 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: WARNING py.warnings [None req-ae2eacf8-24a1-4b5a-80d0-167b210db967 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 28 17:44:21.240594 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: warnings.warn(msg) Jul 28 17:44:21.240594 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: Jul 28 17:44:21.241061 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 790/1556] 10.222.0.89 () {58 vars in 1153 bytes} [Thu Jul 28 17:44:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:44:21.241817 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2eacf8-24a1-4b5a-80d0-167b210db967 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:21.242327 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d907449f-b0b3-41a0-893b-2cfdbd591805 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:21.242618 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d907449f-b0b3-41a0-893b-2cfdbd591805 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:21.242893 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d907449f-b0b3-41a0-893b-2cfdbd591805 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:21.261767 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-d907449f-b0b3-41a0-893b-2cfdbd591805 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:44:21.262971 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-d907449f-b0b3-41a0-893b-2cfdbd591805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:21.272463 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 791/1557] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:44:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:44:21.273302 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d907449f-b0b3-41a0-893b-2cfdbd591805 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:21.282057 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 767/1558] 10.222.0.89 () {60 vars in 1153 bytes} [Thu Jul 28 17:44:21 2022] POST /identity/v3/projects => generated 375 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:44:21.283188 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2eacf8-24a1-4b5a-80d0-167b210db967 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:21.283881 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d907449f-b0b3-41a0-893b-2cfdbd591805 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:21.342922 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7a092a-dc73-480a-b906-f81149c39ad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:21.343490 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7a092a-dc73-480a-b906-f81149c39ad8 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:21.343770 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7a092a-dc73-480a-b906-f81149c39ad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:21.344042 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7a092a-dc73-480a-b906-f81149c39ad8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:21.344736 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7a092a-dc73-480a-b906-f81149c39ad8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:44:21.345057 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7a092a-dc73-480a-b906-f81149c39ad8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:21.346684 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7a092a-dc73-480a-b906-f81149c39ad8 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:21.347672 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2eacf8-24a1-4b5a-80d0-167b210db967 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:21.353642 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7863d809-5a15-4c91-a72d-601b439016bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:21.354311 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7863d809-5a15-4c91-a72d-601b439016bd admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:21.354598 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7863d809-5a15-4c91-a72d-601b439016bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:21.355325 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7863d809-5a15-4c91-a72d-601b439016bd admin admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:21.356062 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7863d809-5a15-4c91-a72d-601b439016bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:21.358131 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 792/1559] 10.222.0.89 () {58 vars in 1153 bytes} [Thu Jul 28 17:44:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:44:21.358448 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7863d809-5a15-4c91-a72d-601b439016bd admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:21.359343 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7a092a-dc73-480a-b906-f81149c39ad8 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:21.375987 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7a092a-dc73-480a-b906-f81149c39ad8 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:21.380922 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399e9289-e563-4530-b866-b23592f42918 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:21.381607 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399e9289-e563-4530-b866-b23592f42918 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:21.381894 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399e9289-e563-4530-b866-b23592f42918 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:21.382164 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399e9289-e563-4530-b866-b23592f42918 admin admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:21.382860 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399e9289-e563-4530-b866-b23592f42918 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:44:21.383162 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399e9289-e563-4530-b866-b23592f42918 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:21.384999 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399e9289-e563-4530-b866-b23592f42918 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:21.394349 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 768/1560] 10.222.0.89 () {60 vars in 1153 bytes} [Thu Jul 28 17:44:21 2022] POST /identity/v3/projects => generated 385 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:44:21.394349 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0fa8eb-f506-4108-ad41-26785f177ada None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:21.394349 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0fa8eb-f506-4108-ad41-26785f177ada None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:21.394349 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0fa8eb-f506-4108-ad41-26785f177ada None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:21.400690 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 793/1561] 10.222.0.89 () {58 vars in 1153 bytes} [Thu Jul 28 17:44:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:44:21.401690 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399e9289-e563-4530-b866-b23592f42918 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:21.411973 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-9e0fa8eb-f506-4108-ad41-26785f177ada None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:44:21.413223 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-9e0fa8eb-f506-4108-ad41-26785f177ada None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:21.416291 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399e9289-e563-4530-b866-b23592f42918 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:21.462943 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:44:21.477332 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5263545-ee9e-4a9a-ab32-7e1508ed8630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:21.477984 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5263545-ee9e-4a9a-ab32-7e1508ed8630 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:21.478264 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5263545-ee9e-4a9a-ab32-7e1508ed8630 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:21.478538 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5263545-ee9e-4a9a-ab32-7e1508ed8630 admin admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:21.479510 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5263545-ee9e-4a9a-ab32-7e1508ed8630 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:44:21.479820 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5263545-ee9e-4a9a-ab32-7e1508ed8630 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:21.481432 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5263545-ee9e-4a9a-ab32-7e1508ed8630 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:21.485001 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 769/1562] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:44:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:44:21.489013 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:44:21.489568 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0fa8eb-f506-4108-ad41-26785f177ada None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:21.497552 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 794/1563] 10.222.0.89 () {58 vars in 1153 bytes} [Thu Jul 28 17:44:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:44:21.498504 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5263545-ee9e-4a9a-ab32-7e1508ed8630 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:21.500489 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0fa8eb-f506-4108-ad41-26785f177ada None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:21.505095 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8d637a-5a59-430d-8547-fe4866de9725 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:21.506274 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8d637a-5a59-430d-8547-fe4866de9725 admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:21.506274 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8d637a-5a59-430d-8547-fe4866de9725 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:21.506274 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8d637a-5a59-430d-8547-fe4866de9725 admin admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:21.506937 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8d637a-5a59-430d-8547-fe4866de9725 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:21.509110 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8d637a-5a59-430d-8547-fe4866de9725 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:21.509852 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5263545-ee9e-4a9a-ab32-7e1508ed8630 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:21.514609 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b10142b-0add-4c1d-9b66-b1f6a6728f4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:21.515970 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b10142b-0add-4c1d-9b66-b1f6a6728f4a admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:21.516254 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b10142b-0add-4c1d-9b66-b1f6a6728f4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:21.516526 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b10142b-0add-4c1d-9b66-b1f6a6728f4a admin admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:21.517252 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b10142b-0add-4c1d-9b66-b1f6a6728f4a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:44:21.517558 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b10142b-0add-4c1d-9b66-b1f6a6728f4a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:21.519146 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b10142b-0add-4c1d-9b66-b1f6a6728f4a admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:21.533737 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 795/1564] 10.222.0.89 () {58 vars in 1153 bytes} [Thu Jul 28 17:44:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:44:21.533737 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b10142b-0add-4c1d-9b66-b1f6a6728f4a admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:21.541430 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 770/1565] 10.222.0.89 () {60 vars in 1153 bytes} [Thu Jul 28 17:44:21 2022] POST /identity/v3/projects => generated 379 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:44:21.541430 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8d637a-5a59-430d-8547-fe4866de9725 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:21.546977 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b10142b-0add-4c1d-9b66-b1f6a6728f4a admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:21.603149 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8d637a-5a59-430d-8547-fe4866de9725 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:21.606740 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16b22d0-2c46-4ad3-9bfa-6417b74989af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:21.607407 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16b22d0-2c46-4ad3-9bfa-6417b74989af admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:21.607685 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86db1fd9-66ae-4320-a1a3-55e9513f9d10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:21.607817 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16b22d0-2c46-4ad3-9bfa-6417b74989af admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:21.607960 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16b22d0-2c46-4ad3-9bfa-6417b74989af admin admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:21.608301 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86db1fd9-66ae-4320-a1a3-55e9513f9d10 admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:21.608602 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86db1fd9-66ae-4320-a1a3-55e9513f9d10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:21.608836 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16b22d0-2c46-4ad3-9bfa-6417b74989af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:21.609325 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86db1fd9-66ae-4320-a1a3-55e9513f9d10 admin admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:21.609634 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86db1fd9-66ae-4320-a1a3-55e9513f9d10 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:21.609790 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: WARNING py.warnings [None req-d16b22d0-2c46-4ad3-9bfa-6417b74989af admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 28 17:44:21.609790 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: warnings.warn(msg) Jul 28 17:44:21.609790 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: Jul 28 17:44:21.611053 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16b22d0-2c46-4ad3-9bfa-6417b74989af admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:21.611516 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86db1fd9-66ae-4320-a1a3-55e9513f9d10 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:21.637905 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 796/1566] 10.222.0.89 () {60 vars in 1144 bytes} [Thu Jul 28 17:44:21 2022] POST /identity/v3/users => generated 288 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:44:21.637905 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16b22d0-2c46-4ad3-9bfa-6417b74989af admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:21.642646 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 771/1567] 10.222.0.89 () {60 vars in 1153 bytes} [Thu Jul 28 17:44:21 2022] POST /identity/v3/projects => generated 377 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:44:21.642646 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86db1fd9-66ae-4320-a1a3-55e9513f9d10 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:21.648901 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16b22d0-2c46-4ad3-9bfa-6417b74989af admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:21.703870 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86db1fd9-66ae-4320-a1a3-55e9513f9d10 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:21.705037 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a0bcba-13bb-430c-bd45-be5a4249020b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:21.705678 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a0bcba-13bb-430c-bd45-be5a4249020b admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:21.705958 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a0bcba-13bb-430c-bd45-be5a4249020b admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:21.706229 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a0bcba-13bb-430c-bd45-be5a4249020b admin admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:21.706883 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a0bcba-13bb-430c-bd45-be5a4249020b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:21.708797 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a0bcba-13bb-430c-bd45-be5a4249020b admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:21.708900 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fd26e4-394f-4bf4-9b23-18093bfc9d6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:21.709550 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fd26e4-394f-4bf4-9b23-18093bfc9d6e admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:21.709833 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fd26e4-394f-4bf4-9b23-18093bfc9d6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:21.710107 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fd26e4-394f-4bf4-9b23-18093bfc9d6e admin admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:21.710767 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fd26e4-394f-4bf4-9b23-18093bfc9d6e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:21.712634 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fd26e4-394f-4bf4-9b23-18093bfc9d6e admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:21.737396 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 797/1568] 10.222.0.89 () {60 vars in 1144 bytes} [Thu Jul 28 17:44:21 2022] POST /identity/v3/users => generated 293 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:44:21.737396 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a0bcba-13bb-430c-bd45-be5a4249020b admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:21.746203 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 772/1569] 10.222.0.89 () {60 vars in 1153 bytes} [Thu Jul 28 17:44:21 2022] POST /identity/v3/projects => generated 391 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:44:21.746203 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fd26e4-394f-4bf4-9b23-18093bfc9d6e admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:21.746487 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a0bcba-13bb-430c-bd45-be5a4249020b admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:21.792270 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:44:21.810583 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fd26e4-394f-4bf4-9b23-18093bfc9d6e admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:21.816001 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb87d53-77e0-4aa9-ae66-4619ecdd5827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:21.816623 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb87d53-77e0-4aa9-ae66-4619ecdd5827 admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:21.816896 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb87d53-77e0-4aa9-ae66-4619ecdd5827 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:21.817230 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb87d53-77e0-4aa9-ae66-4619ecdd5827 admin admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:21.817905 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb87d53-77e0-4aa9-ae66-4619ecdd5827 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:21.819806 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb87d53-77e0-4aa9-ae66-4619ecdd5827 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:21.823923 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885eb593-8b13-473d-84d0-0d5631a5c4bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:21.824558 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885eb593-8b13-473d-84d0-0d5631a5c4bb admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:21.824836 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885eb593-8b13-473d-84d0-0d5631a5c4bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:21.825106 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885eb593-8b13-473d-84d0-0d5631a5c4bb admin admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:21.825840 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885eb593-8b13-473d-84d0-0d5631a5c4bb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:44:21.826250 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885eb593-8b13-473d-84d0-0d5631a5c4bb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:21.828754 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885eb593-8b13-473d-84d0-0d5631a5c4bb admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:21.832036 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:44:21.844893 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 798/1570] 10.222.0.89 () {58 vars in 1153 bytes} [Thu Jul 28 17:44:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:44:21.845826 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885eb593-8b13-473d-84d0-0d5631a5c4bb admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:21.857721 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 773/1571] 10.222.0.89 () {60 vars in 1153 bytes} [Thu Jul 28 17:44:21 2022] POST /identity/v3/projects => generated 375 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:44:21.857721 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb87d53-77e0-4aa9-ae66-4619ecdd5827 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:21.859437 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885eb593-8b13-473d-84d0-0d5631a5c4bb admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:21.916837 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cba9f2-1bc8-4d5d-9059-f8626b2b86ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:21.917605 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cba9f2-1bc8-4d5d-9059-f8626b2b86ab admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:21.917751 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cba9f2-1bc8-4d5d-9059-f8626b2b86ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:21.918027 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cba9f2-1bc8-4d5d-9059-f8626b2b86ab admin admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:21.918366 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb87d53-77e0-4aa9-ae66-4619ecdd5827 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:21.918804 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cba9f2-1bc8-4d5d-9059-f8626b2b86ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:21.920585 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cba9f2-1bc8-4d5d-9059-f8626b2b86ab admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:21.924181 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1ff8ba-f53c-49ad-979c-3634cc277e5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:21.924830 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1ff8ba-f53c-49ad-979c-3634cc277e5e admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:21.925113 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1ff8ba-f53c-49ad-979c-3634cc277e5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:21.925420 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1ff8ba-f53c-49ad-979c-3634cc277e5e admin admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:21.926096 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1ff8ba-f53c-49ad-979c-3634cc277e5e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:21.928033 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1ff8ba-f53c-49ad-979c-3634cc277e5e admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:21.949680 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 799/1572] 10.222.0.89 () {60 vars in 1153 bytes} [Thu Jul 28 17:44:21 2022] POST /identity/v3/projects => generated 389 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:44:21.950650 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cba9f2-1bc8-4d5d-9059-f8626b2b86ab admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:21.952066 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 774/1573] 10.222.0.89 () {60 vars in 1144 bytes} [Thu Jul 28 17:44:21 2022] POST /identity/v3/users => generated 290 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:44:21.952911 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1ff8ba-f53c-49ad-979c-3634cc277e5e admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:21.963224 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1ff8ba-f53c-49ad-979c-3634cc277e5e admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:22.014504 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cba9f2-1bc8-4d5d-9059-f8626b2b86ab admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:22.019431 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0bb80c-b27c-4bab-8c2d-18078a81b707 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:22.020049 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0bb80c-b27c-4bab-8c2d-18078a81b707 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:22.020335 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0bb80c-b27c-4bab-8c2d-18078a81b707 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:22.020610 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0bb80c-b27c-4bab-8c2d-18078a81b707 admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:22.021482 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0bb80c-b27c-4bab-8c2d-18078a81b707 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:22.023082 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0bb80c-b27c-4bab-8c2d-18078a81b707 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:22.024910 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be0c818-589d-4894-bf89-c009cfea058e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:22.025565 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be0c818-589d-4894-bf89-c009cfea058e admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:22.025898 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be0c818-589d-4894-bf89-c009cfea058e admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:22.026172 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be0c818-589d-4894-bf89-c009cfea058e admin admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:22.026963 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be0c818-589d-4894-bf89-c009cfea058e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:22.028966 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be0c818-589d-4894-bf89-c009cfea058e admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:22.034350 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 800/1574] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:44:21 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:44:22.035386 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0bb80c-b27c-4bab-8c2d-18078a81b707 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:22.045918 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0bb80c-b27c-4bab-8c2d-18078a81b707 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:22.051907 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0849e48a-9376-413a-8607-5a3ddc614cb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:22.052538 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0849e48a-9376-413a-8607-5a3ddc614cb2 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:22.052816 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0849e48a-9376-413a-8607-5a3ddc614cb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:22.053086 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0849e48a-9376-413a-8607-5a3ddc614cb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:22.053804 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0849e48a-9376-413a-8607-5a3ddc614cb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:22.057128 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 775/1575] 10.222.0.89 () {60 vars in 1144 bytes} [Thu Jul 28 17:44:21 2022] POST /identity/v3/users => generated 289 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:44:22.057128 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be0c818-589d-4894-bf89-c009cfea058e admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:22.057385 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0849e48a-9376-413a-8607-5a3ddc614cb2 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:22.065182 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be0c818-589d-4894-bf89-c009cfea058e admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:22.067944 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 801/1576] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:44:22 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:44:22.068709 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0849e48a-9376-413a-8607-5a3ddc614cb2 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:22.071051 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03b8e71-a9b4-4ee3-a7fb-217f79a562ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:22.071670 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03b8e71-a9b4-4ee3-a7fb-217f79a562ac admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:22.071952 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03b8e71-a9b4-4ee3-a7fb-217f79a562ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:22.072228 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03b8e71-a9b4-4ee3-a7fb-217f79a562ac admin admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:22.072931 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03b8e71-a9b4-4ee3-a7fb-217f79a562ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:22.074992 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03b8e71-a9b4-4ee3-a7fb-217f79a562ac admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:22.079057 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0849e48a-9376-413a-8607-5a3ddc614cb2 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:22.083729 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6d4f0e-80d8-41d1-b8c1-b7ad1109af7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:22.084350 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6d4f0e-80d8-41d1-b8c1-b7ad1109af7f admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:22.084625 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6d4f0e-80d8-41d1-b8c1-b7ad1109af7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:22.084894 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6d4f0e-80d8-41d1-b8c1-b7ad1109af7f admin admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:22.085585 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6d4f0e-80d8-41d1-b8c1-b7ad1109af7f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:22.087538 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6d4f0e-80d8-41d1-b8c1-b7ad1109af7f admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:22.100551 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 776/1577] 10.222.0.89 () {60 vars in 1144 bytes} [Thu Jul 28 17:44:22 2022] POST /identity/v3/users => generated 296 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:44:22.100551 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03b8e71-a9b4-4ee3-a7fb-217f79a562ac admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:22.110330 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03b8e71-a9b4-4ee3-a7fb-217f79a562ac admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:22.115470 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d869862-bce0-4824-9731-171cbbbe1cde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:22.116089 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d869862-bce0-4824-9731-171cbbbe1cde admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:22.116374 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d869862-bce0-4824-9731-171cbbbe1cde admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:22.116650 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d869862-bce0-4824-9731-171cbbbe1cde admin admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:22.117373 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d869862-bce0-4824-9731-171cbbbe1cde admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:22.118093 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 802/1578] 10.222.0.89 () {60 vars in 1153 bytes} [Thu Jul 28 17:44:22 2022] POST /identity/v3/projects => generated 405 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:44:22.118924 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6d4f0e-80d8-41d1-b8c1-b7ad1109af7f admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:22.119452 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d869862-bce0-4824-9731-171cbbbe1cde admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:22.149831 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 777/1579] 10.222.0.89 () {60 vars in 1144 bytes} [Thu Jul 28 17:44:22 2022] POST /identity/v3/users => generated 288 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:44:22.150977 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d869862-bce0-4824-9731-171cbbbe1cde admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:22.164613 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d869862-bce0-4824-9731-171cbbbe1cde admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:22.172636 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:44:22.181089 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=87,keystone:INSERT=13 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:44:22.197050 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6d4f0e-80d8-41d1-b8c1-b7ad1109af7f admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:22.202012 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c73e783-b3f1-4e53-b36c-e2196d6b8970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:22.202663 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c73e783-b3f1-4e53-b36c-e2196d6b8970 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:22.202949 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c73e783-b3f1-4e53-b36c-e2196d6b8970 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:22.203226 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c73e783-b3f1-4e53-b36c-e2196d6b8970 admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:22.203920 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c73e783-b3f1-4e53-b36c-e2196d6b8970 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:22.205590 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c73e783-b3f1-4e53-b36c-e2196d6b8970 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:22.215565 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 803/1580] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:44:22 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:44:22.216373 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c73e783-b3f1-4e53-b36c-e2196d6b8970 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:22.226909 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c73e783-b3f1-4e53-b36c-e2196d6b8970 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:22.231598 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2202104b-0dfb-4d61-8a98-a0cd17416f9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:22.232196 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2202104b-0dfb-4d61-8a98-a0cd17416f9f admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:22.232478 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2202104b-0dfb-4d61-8a98-a0cd17416f9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:22.232750 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2202104b-0dfb-4d61-8a98-a0cd17416f9f admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:22.233465 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2202104b-0dfb-4d61-8a98-a0cd17416f9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:22.235699 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2202104b-0dfb-4d61-8a98-a0cd17416f9f admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:22.236933 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2622afda-37b9-4cf4-ba6a-75d566921e08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:22.237473 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2622afda-37b9-4cf4-ba6a-75d566921e08 admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:22.237758 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2622afda-37b9-4cf4-ba6a-75d566921e08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:22.238035 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2622afda-37b9-4cf4-ba6a-75d566921e08 admin admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:22.238707 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2622afda-37b9-4cf4-ba6a-75d566921e08 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:22.240621 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2622afda-37b9-4cf4-ba6a-75d566921e08 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:22.247422 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 804/1581] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:44:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:44:22.248376 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2202104b-0dfb-4d61-8a98-a0cd17416f9f admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:22.259459 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2202104b-0dfb-4d61-8a98-a0cd17416f9f admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:22.264316 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c7c4cd-bf75-48f1-a593-8b4c2f562073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:22.264805 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 778/1582] 10.222.0.89 () {60 vars in 1144 bytes} [Thu Jul 28 17:44:22 2022] POST /identity/v3/users => generated 295 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:44:22.265026 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c7c4cd-bf75-48f1-a593-8b4c2f562073 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:22.265321 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c7c4cd-bf75-48f1-a593-8b4c2f562073 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:22.265562 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2622afda-37b9-4cf4-ba6a-75d566921e08 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:22.265680 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c7c4cd-bf75-48f1-a593-8b4c2f562073 admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:22.266424 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c7c4cd-bf75-48f1-a593-8b4c2f562073 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:22.268186 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c7c4cd-bf75-48f1-a593-8b4c2f562073 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:22.277899 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2622afda-37b9-4cf4-ba6a-75d566921e08 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:22.281839 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 805/1583] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:44:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:44:22.281839 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c7c4cd-bf75-48f1-a593-8b4c2f562073 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:22.284020 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e5c678-8c29-47c4-b29a-d717623ce355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:22.284655 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e5c678-8c29-47c4-b29a-d717623ce355 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:22.284973 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e5c678-8c29-47c4-b29a-d717623ce355 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:22.285293 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e5c678-8c29-47c4-b29a-d717623ce355 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:22.286074 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e5c678-8c29-47c4-b29a-d717623ce355 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:22.287683 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e5c678-8c29-47c4-b29a-d717623ce355 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:22.290740 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c7c4cd-bf75-48f1-a593-8b4c2f562073 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:22.295790 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f277c4-84da-4de0-8194-e00fe10f0095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:22.297018 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f277c4-84da-4de0-8194-e00fe10f0095 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:22.297331 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f277c4-84da-4de0-8194-e00fe10f0095 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:22.297859 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f277c4-84da-4de0-8194-e00fe10f0095 admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:22.298566 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f277c4-84da-4de0-8194-e00fe10f0095 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:22.302169 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 779/1584] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:44:22 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:44:22.302169 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e5c678-8c29-47c4-b29a-d717623ce355 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:22.302415 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f277c4-84da-4de0-8194-e00fe10f0095 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:22.310054 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e5c678-8c29-47c4-b29a-d717623ce355 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:22.310579 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 806/1585] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:44:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:44:22.311459 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f277c4-84da-4de0-8194-e00fe10f0095 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:22.315984 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec1a08b-bd66-40d2-993b-4f0afead08b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:22.316609 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec1a08b-bd66-40d2-993b-4f0afead08b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:22.317116 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec1a08b-bd66-40d2-993b-4f0afead08b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:22.317428 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec1a08b-bd66-40d2-993b-4f0afead08b1 admin admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:22.318102 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec1a08b-bd66-40d2-993b-4f0afead08b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:22.320028 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec1a08b-bd66-40d2-993b-4f0afead08b1 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:22.321916 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f277c4-84da-4de0-8194-e00fe10f0095 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:22.326511 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd23d844-1cac-4c6f-8b3a-6c5a1cd905bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:22.327110 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd23d844-1cac-4c6f-8b3a-6c5a1cd905bb admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:22.327413 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd23d844-1cac-4c6f-8b3a-6c5a1cd905bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:22.327685 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd23d844-1cac-4c6f-8b3a-6c5a1cd905bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:22.328371 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd23d844-1cac-4c6f-8b3a-6c5a1cd905bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:22.330063 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd23d844-1cac-4c6f-8b3a-6c5a1cd905bb admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:22.340712 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 807/1586] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:44:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:44:22.341610 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd23d844-1cac-4c6f-8b3a-6c5a1cd905bb admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:22.344676 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 780/1587] 10.222.0.89 () {60 vars in 1144 bytes} [Thu Jul 28 17:44:22 2022] POST /identity/v3/users => generated 303 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:44:22.345645 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec1a08b-bd66-40d2-993b-4f0afead08b1 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:22.352525 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd23d844-1cac-4c6f-8b3a-6c5a1cd905bb admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:22.355233 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec1a08b-bd66-40d2-993b-4f0afead08b1 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:22.357364 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721ff2cf-102f-482b-9844-7fecd074f4e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:22.357970 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721ff2cf-102f-482b-9844-7fecd074f4e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:22.358253 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721ff2cf-102f-482b-9844-7fecd074f4e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:22.358528 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721ff2cf-102f-482b-9844-7fecd074f4e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:22.359860 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721ff2cf-102f-482b-9844-7fecd074f4e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:22.360908 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89f4b3f-07d1-45b2-a4c2-1087613b5927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:22.361495 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721ff2cf-102f-482b-9844-7fecd074f4e7 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:22.361610 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89f4b3f-07d1-45b2-a4c2-1087613b5927 admin admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:22.361813 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89f4b3f-07d1-45b2-a4c2-1087613b5927 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:22.362090 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89f4b3f-07d1-45b2-a4c2-1087613b5927 admin admin] PATH_INFO: `/v3/projects/78a2f25af7784086aadec8dd5a1015dd/users/22479a66b0f34038b144f4def1aa3f43/roles/11f8731756fe4caca13efa61b71b11c5` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:22.371838 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 808/1588] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:44:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:44:22.372648 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721ff2cf-102f-482b-9844-7fecd074f4e7 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:22.377327 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89f4b3f-07d1-45b2-a4c2-1087613b5927 admin admin] RBAC: Authorizing `identity:create_grant(project_id=78a2f25af7784086aadec8dd5a1015dd, user_id=22479a66b0f34038b144f4def1aa3f43, role_id=11f8731756fe4caca13efa61b71b11c5)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:22.378377 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: WARNING py.warnings [None req-d89f4b3f-07d1-45b2-a4c2-1087613b5927 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 28 17:44:22.378377 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: warnings.warn(msg) Jul 28 17:44:22.378377 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: Jul 28 17:44:22.380334 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89f4b3f-07d1-45b2-a4c2-1087613b5927 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:22.382625 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721ff2cf-102f-482b-9844-7fecd074f4e7 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:22.388797 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae335d53-c7b8-4a01-b97e-8385bf02c80d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:22.389512 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae335d53-c7b8-4a01-b97e-8385bf02c80d admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:22.389796 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae335d53-c7b8-4a01-b97e-8385bf02c80d admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:22.390658 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae335d53-c7b8-4a01-b97e-8385bf02c80d admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:22.391358 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae335d53-c7b8-4a01-b97e-8385bf02c80d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:22.393016 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae335d53-c7b8-4a01-b97e-8385bf02c80d admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:22.395514 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 781/1589] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:44:22 2022] PUT /identity/v3/projects/78a2f25af7784086aadec8dd5a1015dd/users/22479a66b0f34038b144f4def1aa3f43/roles/11f8731756fe4caca13efa61b71b11c5 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:44:22.399840 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89f4b3f-07d1-45b2-a4c2-1087613b5927 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:22.403177 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 809/1590] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:44:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:44:22.403952 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae335d53-c7b8-4a01-b97e-8385bf02c80d admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:22.414715 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae335d53-c7b8-4a01-b97e-8385bf02c80d admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:22.479257 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d41d904-23c5-41d9-a802-4d00df2b6526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:22.479257 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d41d904-23c5-41d9-a802-4d00df2b6526 admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:22.479257 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d41d904-23c5-41d9-a802-4d00df2b6526 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:22.479257 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d41d904-23c5-41d9-a802-4d00df2b6526 admin admin] PATH_INFO: `/v3/projects/1740f22f19344b8a91769cc0881dad7b/users/ae0586e81d7e477ea9ae3efa55c8a4da/roles/11f8731756fe4caca13efa61b71b11c5` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:22.479776 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89f4b3f-07d1-45b2-a4c2-1087613b5927 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:22.482373 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fc9864-7b3c-43c4-9539-f521b6ef574f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:22.482994 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fc9864-7b3c-43c4-9539-f521b6ef574f admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:22.483270 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fc9864-7b3c-43c4-9539-f521b6ef574f admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:22.483536 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fc9864-7b3c-43c4-9539-f521b6ef574f admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:22.485020 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fc9864-7b3c-43c4-9539-f521b6ef574f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:22.486606 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fc9864-7b3c-43c4-9539-f521b6ef574f admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:22.496260 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d41d904-23c5-41d9-a802-4d00df2b6526 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1740f22f19344b8a91769cc0881dad7b, user_id=ae0586e81d7e477ea9ae3efa55c8a4da, role_id=11f8731756fe4caca13efa61b71b11c5)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:22.498943 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 782/1591] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:44:22 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:44:22.499062 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d41d904-23c5-41d9-a802-4d00df2b6526 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:22.499825 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fc9864-7b3c-43c4-9539-f521b6ef574f admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:22.510795 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fc9864-7b3c-43c4-9539-f521b6ef574f admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:22.514522 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 810/1592] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:44:22 2022] PUT /identity/v3/projects/1740f22f19344b8a91769cc0881dad7b/users/ae0586e81d7e477ea9ae3efa55c8a4da/roles/11f8731756fe4caca13efa61b71b11c5 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:44:22.516471 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d41d904-23c5-41d9-a802-4d00df2b6526 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:22.568464 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:44:22.574501 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:44:22.579559 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d41d904-23c5-41d9-a802-4d00df2b6526 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:22.583849 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d11869d-27bb-447f-9370-bd4c40fb208c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:22.584617 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d11869d-27bb-447f-9370-bd4c40fb208c admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:22.584899 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d11869d-27bb-447f-9370-bd4c40fb208c admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:22.585254 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d11869d-27bb-447f-9370-bd4c40fb208c admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:22.585501 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d313a005-9080-4f2b-bf55-220d22056b71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:22.585890 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d11869d-27bb-447f-9370-bd4c40fb208c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:22.586124 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d313a005-9080-4f2b-bf55-220d22056b71 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:22.586472 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d313a005-9080-4f2b-bf55-220d22056b71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:22.586732 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d313a005-9080-4f2b-bf55-220d22056b71 admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:22.587427 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d313a005-9080-4f2b-bf55-220d22056b71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:22.587561 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d11869d-27bb-447f-9370-bd4c40fb208c admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:22.589087 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d313a005-9080-4f2b-bf55-220d22056b71 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:22.598639 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 783/1593] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:44:22 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:44:22.599455 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d11869d-27bb-447f-9370-bd4c40fb208c admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:22.603073 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 811/1594] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:44:22 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:44:22.603073 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d313a005-9080-4f2b-bf55-220d22056b71 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:22.611099 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d313a005-9080-4f2b-bf55-220d22056b71 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:22.611354 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d11869d-27bb-447f-9370-bd4c40fb208c admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:22.615669 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c33d92-8460-42e3-8f50-ef39d0f12cc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:22.616266 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdfc0f2-a0d3-4692-a2b1-c12cffa3dd81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:22.616428 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c33d92-8460-42e3-8f50-ef39d0f12cc4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:22.616576 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c33d92-8460-42e3-8f50-ef39d0f12cc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:22.616849 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c33d92-8460-42e3-8f50-ef39d0f12cc4 admin admin] PATH_INFO: `/v3/projects/bdc42a1f064b4cf0acb0d9ce3dee6b94/users/3586e07a652b4f6f93c9448d9d1db885/roles/11f8731756fe4caca13efa61b71b11c5` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:22.616946 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdfc0f2-a0d3-4692-a2b1-c12cffa3dd81 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:22.617291 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdfc0f2-a0d3-4692-a2b1-c12cffa3dd81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:22.617568 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdfc0f2-a0d3-4692-a2b1-c12cffa3dd81 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:22.618381 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdfc0f2-a0d3-4692-a2b1-c12cffa3dd81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:22.620086 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdfc0f2-a0d3-4692-a2b1-c12cffa3dd81 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:22.630499 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 784/1595] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:44:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:44:22.631729 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a7df5f-ef26-4a8b-9a61-2b3469f5806a None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:22.632018 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a7df5f-ef26-4a8b-9a61-2b3469f5806a None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:22.632294 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a7df5f-ef26-4a8b-9a61-2b3469f5806a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:22.632413 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c33d92-8460-42e3-8f50-ef39d0f12cc4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bdc42a1f064b4cf0acb0d9ce3dee6b94, user_id=3586e07a652b4f6f93c9448d9d1db885, role_id=11f8731756fe4caca13efa61b71b11c5)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:22.635445 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c33d92-8460-42e3-8f50-ef39d0f12cc4 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:22.650304 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 812/1596] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:44:22 2022] PUT /identity/v3/projects/bdc42a1f064b4cf0acb0d9ce3dee6b94/users/3586e07a652b4f6f93c9448d9d1db885/roles/11f8731756fe4caca13efa61b71b11c5 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:44:22.652113 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-b7a7df5f-ef26-4a8b-9a61-2b3469f5806a None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:44:22.653349 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-b7a7df5f-ef26-4a8b-9a61-2b3469f5806a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:22.658297 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c33d92-8460-42e3-8f50-ef39d0f12cc4 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:22.713794 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 785/1597] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:44:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:44:22.714563 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a7df5f-ef26-4a8b-9a61-2b3469f5806a None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:22.721994 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c33d92-8460-42e3-8f50-ef39d0f12cc4 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:22.727483 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a7df5f-ef26-4a8b-9a61-2b3469f5806a None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:22.727666 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efccc3ba-b997-4e6d-aed4-3a357ebf7a12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:22.728283 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efccc3ba-b997-4e6d-aed4-3a357ebf7a12 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:22.728565 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efccc3ba-b997-4e6d-aed4-3a357ebf7a12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:22.728844 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efccc3ba-b997-4e6d-aed4-3a357ebf7a12 admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:22.729581 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efccc3ba-b997-4e6d-aed4-3a357ebf7a12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:22.731183 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efccc3ba-b997-4e6d-aed4-3a357ebf7a12 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:22.732536 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c315b8-1620-4fd6-b37f-e0e779198861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:22.733574 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c315b8-1620-4fd6-b37f-e0e779198861 admin admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:22.733574 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c315b8-1620-4fd6-b37f-e0e779198861 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:22.734069 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c315b8-1620-4fd6-b37f-e0e779198861 admin admin] PATH_INFO: `/v3/projects/6018db8e046e4945be4dd56ff8e8eeea/users/6ed334d56f34486ebd8b52247d060856/roles/11f8731756fe4caca13efa61b71b11c5` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:22.742000 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 813/1598] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:44:22 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:44:22.743263 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16594dbb-b662-48e3-9edd-c8a00540b63f None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:22.743553 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16594dbb-b662-48e3-9edd-c8a00540b63f None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:22.743831 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16594dbb-b662-48e3-9edd-c8a00540b63f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:22.752517 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c315b8-1620-4fd6-b37f-e0e779198861 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6018db8e046e4945be4dd56ff8e8eeea, user_id=6ed334d56f34486ebd8b52247d060856, role_id=11f8731756fe4caca13efa61b71b11c5)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:22.755240 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c315b8-1620-4fd6-b37f-e0e779198861 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:22.770377 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 786/1599] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:44:22 2022] PUT /identity/v3/projects/6018db8e046e4945be4dd56ff8e8eeea/users/6ed334d56f34486ebd8b52247d060856/roles/11f8731756fe4caca13efa61b71b11c5 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:44:22.771167 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c315b8-1620-4fd6-b37f-e0e779198861 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:22.778318 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-16594dbb-b662-48e3-9edd-c8a00540b63f None None] MFA Rules not processed for user `ae0586e81d7e477ea9ae3efa55c8a4da`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:44:22.780203 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-16594dbb-b662-48e3-9edd-c8a00540b63f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:22.840888 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c315b8-1620-4fd6-b37f-e0e779198861 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:22.846648 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e3f680-8e3f-48c0-8aba-9ebfbdd3cbb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:22.847554 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e3f680-8e3f-48c0-8aba-9ebfbdd3cbb4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:22.847959 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e3f680-8e3f-48c0-8aba-9ebfbdd3cbb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:22.848333 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e3f680-8e3f-48c0-8aba-9ebfbdd3cbb4 admin admin] PATH_INFO: `/v3/projects/f5332e245b754728b04af219fdb6ca93/users/14327b8153ab4de782d4bcfef7681fc1/roles/11f8731756fe4caca13efa61b71b11c5` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:22.851199 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 814/1600] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:44:22 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:44:22.852756 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16594dbb-b662-48e3-9edd-c8a00540b63f None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:22.866754 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e3f680-8e3f-48c0-8aba-9ebfbdd3cbb4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5332e245b754728b04af219fdb6ca93, user_id=14327b8153ab4de782d4bcfef7681fc1, role_id=11f8731756fe4caca13efa61b71b11c5)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:22.867618 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16594dbb-b662-48e3-9edd-c8a00540b63f None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:22.869617 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e3f680-8e3f-48c0-8aba-9ebfbdd3cbb4 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:22.875656 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2f54e7-a57b-4e5d-9aa2-b8e674d11e1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:22.876442 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2f54e7-a57b-4e5d-9aa2-b8e674d11e1a admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:22.876953 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2f54e7-a57b-4e5d-9aa2-b8e674d11e1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:22.877440 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2f54e7-a57b-4e5d-9aa2-b8e674d11e1a admin admin] PATH_INFO: `/v3/projects/29887ef23f954dc18aea8bd7e847cabc/users/0d61a50992b241efbad5af3ece48a8ea/roles/11f8731756fe4caca13efa61b71b11c5` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:22.889891 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 787/1601] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:44:22 2022] PUT /identity/v3/projects/f5332e245b754728b04af219fdb6ca93/users/14327b8153ab4de782d4bcfef7681fc1/roles/11f8731756fe4caca13efa61b71b11c5 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:44:22.889891 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e3f680-8e3f-48c0-8aba-9ebfbdd3cbb4 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:22.904230 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2f54e7-a57b-4e5d-9aa2-b8e674d11e1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=29887ef23f954dc18aea8bd7e847cabc, user_id=0d61a50992b241efbad5af3ece48a8ea, role_id=11f8731756fe4caca13efa61b71b11c5)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:22.910453 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2f54e7-a57b-4e5d-9aa2-b8e674d11e1a admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:22.927781 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:44:22.928654 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 815/1602] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:44:22 2022] PUT /identity/v3/projects/29887ef23f954dc18aea8bd7e847cabc/users/0d61a50992b241efbad5af3ece48a8ea/roles/11f8731756fe4caca13efa61b71b11c5 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:44:22.930805 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f18c7a9-7ae9-4dd0-97cd-29b521b011ff None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:22.931302 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f18c7a9-7ae9-4dd0-97cd-29b521b011ff None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:22.931772 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f18c7a9-7ae9-4dd0-97cd-29b521b011ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:22.963820 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-6f18c7a9-7ae9-4dd0-97cd-29b521b011ff None None] MFA Rules not processed for user `3586e07a652b4f6f93c9448d9d1db885`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:44:22.964084 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e3f680-8e3f-48c0-8aba-9ebfbdd3cbb4 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:22.965204 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-6f18c7a9-7ae9-4dd0-97cd-29b521b011ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:22.985846 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:44:23.026097 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272808bb-2002-4d8a-bc9c-f33b6186766b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:23.026728 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272808bb-2002-4d8a-bc9c-f33b6186766b admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:23.027012 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272808bb-2002-4d8a-bc9c-f33b6186766b admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:23.027288 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272808bb-2002-4d8a-bc9c-f33b6186766b admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:23.027987 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272808bb-2002-4d8a-bc9c-f33b6186766b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:23.029972 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272808bb-2002-4d8a-bc9c-f33b6186766b admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:23.034923 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 816/1603] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:44:22 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:44:23.038556 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f18c7a9-7ae9-4dd0-97cd-29b521b011ff None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:23.041142 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 788/1604] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:44:22 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:44:23.042084 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272808bb-2002-4d8a-bc9c-f33b6186766b admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:23.055538 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272808bb-2002-4d8a-bc9c-f33b6186766b admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:23.060331 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2c77ba-40d8-4eed-a029-0f12e4fd3e1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:23.061001 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2c77ba-40d8-4eed-a029-0f12e4fd3e1e admin admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:23.061317 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2c77ba-40d8-4eed-a029-0f12e4fd3e1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:23.061598 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2c77ba-40d8-4eed-a029-0f12e4fd3e1e admin admin] PATH_INFO: `/v3/projects/65535f9817c448dfac997e93efa1a909/users/0b97c60c0360466fb878e1e9141e1f3e/roles/11f8731756fe4caca13efa61b71b11c5` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:23.076207 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2c77ba-40d8-4eed-a029-0f12e4fd3e1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=65535f9817c448dfac997e93efa1a909, user_id=0b97c60c0360466fb878e1e9141e1f3e, role_id=11f8731756fe4caca13efa61b71b11c5)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:23.079060 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2c77ba-40d8-4eed-a029-0f12e4fd3e1e admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:23.092487 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f18c7a9-7ae9-4dd0-97cd-29b521b011ff None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:23.092683 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 789/1605] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:44:23 2022] PUT /identity/v3/projects/65535f9817c448dfac997e93efa1a909/users/0b97c60c0360466fb878e1e9141e1f3e/roles/11f8731756fe4caca13efa61b71b11c5 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:44:23.094709 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992dd93d-193b-4ef5-91bb-71f2d34a3979 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:23.095031 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992dd93d-193b-4ef5-91bb-71f2d34a3979 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:23.095450 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992dd93d-193b-4ef5-91bb-71f2d34a3979 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:23.114818 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-992dd93d-193b-4ef5-91bb-71f2d34a3979 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:44:23.116043 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-992dd93d-193b-4ef5-91bb-71f2d34a3979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:23.149229 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce1bc2c-8acf-4e85-a4a0-97ecc5ff6ae0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76817cd541294331b0023069092e8e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:23.149887 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce1bc2c-8acf-4e85-a4a0-97ecc5ff6ae0 service neutron] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:23.150169 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce1bc2c-8acf-4e85-a4a0-97ecc5ff6ae0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:23.150449 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce1bc2c-8acf-4e85-a4a0-97ecc5ff6ae0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:23.177372 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 790/1606] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:44:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:44:23.180943 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa7dc98-c6e9-4739-b31e-ad6ccf4f3033 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:23.180943 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa7dc98-c6e9-4739-b31e-ad6ccf4f3033 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:23.180943 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa7dc98-c6e9-4739-b31e-ad6ccf4f3033 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:23.180943 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 791/1607] 10.222.0.89 () {54 vars in 962 bytes} [Thu Jul 28 17:44:23 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:44:23.181889 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c571278f-4191-4ba2-921c-1dabc3160e2b None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:23.182264 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c571278f-4191-4ba2-921c-1dabc3160e2b None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:23.182643 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c571278f-4191-4ba2-921c-1dabc3160e2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:23.201788 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-c571278f-4191-4ba2-921c-1dabc3160e2b None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:44:23.203561 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-c571278f-4191-4ba2-921c-1dabc3160e2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:23.211332 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 792/1608] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:44:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:44:23.212534 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173dfbc6-00f8-47d1-8336-bffe8ffdbbbb None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:23.212817 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173dfbc6-00f8-47d1-8336-bffe8ffdbbbb None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:23.213089 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173dfbc6-00f8-47d1-8336-bffe8ffdbbbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:23.213897 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce1bc2c-8acf-4e85-a4a0-97ecc5ff6ae0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:23.215505 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce1bc2c-8acf-4e85-a4a0-97ecc5ff6ae0 service neutron] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:23.226856 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 817/1609] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:44:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:44:23.228047 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce1bc2c-8acf-4e85-a4a0-97ecc5ff6ae0 service neutron] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:23.238573 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce1bc2c-8acf-4e85-a4a0-97ecc5ff6ae0 service neutron] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:23.243963 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86df0795-7400-4ae5-b5f3-aa687ade06c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:23.244812 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86df0795-7400-4ae5-b5f3-aa687ade06c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:23.245396 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86df0795-7400-4ae5-b5f3-aa687ade06c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:23.245517 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-173dfbc6-00f8-47d1-8336-bffe8ffdbbbb None None] MFA Rules not processed for user `0d61a50992b241efbad5af3ece48a8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:44:23.245690 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86df0795-7400-4ae5-b5f3-aa687ade06c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:23.246452 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86df0795-7400-4ae5-b5f3-aa687ade06c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:23.246567 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-173dfbc6-00f8-47d1-8336-bffe8ffdbbbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:23.248675 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86df0795-7400-4ae5-b5f3-aa687ade06c7 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:23.282017 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 818/1610] 10.222.0.89 () {60 vars in 1153 bytes} [Thu Jul 28 17:44:23 2022] POST /identity/v3/projects => generated 381 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:44:23.285868 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86df0795-7400-4ae5-b5f3-aa687ade06c7 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:23.286536 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:44:23.291879 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:44:23.334177 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 793/1611] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:44:23 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:44:23.334177 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c64786-49ea-4cf2-9254-337a7ca5fbc0 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:23.334177 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c64786-49ea-4cf2-9254-337a7ca5fbc0 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:23.334177 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c64786-49ea-4cf2-9254-337a7ca5fbc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:23.366381 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-56c64786-49ea-4cf2-9254-337a7ca5fbc0 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:44:23.368436 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-56c64786-49ea-4cf2-9254-337a7ca5fbc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:23.370413 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86df0795-7400-4ae5-b5f3-aa687ade06c7 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:23.377813 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea665f2-686d-46fe-95d3-ffca9dd716a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:23.378605 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea665f2-686d-46fe-95d3-ffca9dd716a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:23.381208 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea665f2-686d-46fe-95d3-ffca9dd716a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:23.381346 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 794/1612] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:44:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:44:23.381436 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea665f2-686d-46fe-95d3-ffca9dd716a1 admin admin] PATH_INFO: `/v3/projects/b708738fea994bae8807b3c0a93a0383/users/4f2ffc8d7e9044d2929416598eee79f6/roles/11f8731756fe4caca13efa61b71b11c5` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:23.381533 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc903bc-c8cd-4ce9-bf19-f8ca162b7680 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:23.381533 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc903bc-c8cd-4ce9-bf19-f8ca162b7680 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:23.382207 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc903bc-c8cd-4ce9-bf19-f8ca162b7680 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:23.401323 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea665f2-686d-46fe-95d3-ffca9dd716a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b708738fea994bae8807b3c0a93a0383, user_id=4f2ffc8d7e9044d2929416598eee79f6, role_id=11f8731756fe4caca13efa61b71b11c5)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:23.404477 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea665f2-686d-46fe-95d3-ffca9dd716a1 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:23.420340 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 819/1613] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:44:23 2022] PUT /identity/v3/projects/b708738fea994bae8807b3c0a93a0383/users/4f2ffc8d7e9044d2929416598eee79f6/roles/11f8731756fe4caca13efa61b71b11c5 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:44:23.421496 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea665f2-686d-46fe-95d3-ffca9dd716a1 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:23.423458 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-fea665f2-686d-46fe-95d3-ffca9dd716a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:23.430381 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-ccc903bc-c8cd-4ce9-bf19-f8ca162b7680 None None] MFA Rules not processed for user `d10b49b29df94404a71f43273839c252`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:44:23.431727 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-ccc903bc-c8cd-4ce9-bf19-f8ca162b7680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:23.499358 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea665f2-686d-46fe-95d3-ffca9dd716a1 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:23.503238 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a61b14-f9e4-423a-96fa-0b8eced2fb9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76817cd541294331b0023069092e8e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:23.503858 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a61b14-f9e4-423a-96fa-0b8eced2fb9d service neutron] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:23.504140 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a61b14-f9e4-423a-96fa-0b8eced2fb9d service neutron] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:23.504633 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a61b14-f9e4-423a-96fa-0b8eced2fb9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:23.509337 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 795/1614] 10.222.0.89 () {58 vars in 1059 bytes} [Thu Jul 28 17:44:23 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:44:23.514360 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc903bc-c8cd-4ce9-bf19-f8ca162b7680 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:23.526732 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc903bc-c8cd-4ce9-bf19-f8ca162b7680 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:23.532269 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de060138-7ddd-41a0-8d86-665e0fc8242a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76817cd541294331b0023069092e8e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:23.532958 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de060138-7ddd-41a0-8d86-665e0fc8242a service neutron] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:23.533271 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de060138-7ddd-41a0-8d86-665e0fc8242a service neutron] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:23.533545 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de060138-7ddd-41a0-8d86-665e0fc8242a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:23.589106 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a61b14-f9e4-423a-96fa-0b8eced2fb9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:23.590833 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a61b14-f9e4-423a-96fa-0b8eced2fb9d service neutron] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:23.602952 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 820/1615] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:44:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:44:23.604361 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a61b14-f9e4-423a-96fa-0b8eced2fb9d service neutron] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:23.615725 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de060138-7ddd-41a0-8d86-665e0fc8242a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:23.616130 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a61b14-f9e4-423a-96fa-0b8eced2fb9d service neutron] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:23.617492 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de060138-7ddd-41a0-8d86-665e0fc8242a service neutron] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:23.630340 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 796/1616] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:44:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:44:23.631828 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2169a5-4a44-4726-ab07-0dfbd3bc6d01 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:23.632904 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2169a5-4a44-4726-ab07-0dfbd3bc6d01 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:23.633537 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2169a5-4a44-4726-ab07-0dfbd3bc6d01 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:23.633666 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac552faf-bbd3-46c5-a9fc-287fecd2b6cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:23.634578 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac552faf-bbd3-46c5-a9fc-287fecd2b6cc admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:23.634893 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac552faf-bbd3-46c5-a9fc-287fecd2b6cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:23.635062 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 797/1617] 10.222.0.89 () {54 vars in 962 bytes} [Thu Jul 28 17:44:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:44:23.635932 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2169a5-4a44-4726-ab07-0dfbd3bc6d01 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:23.641536 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac552faf-bbd3-46c5-a9fc-287fecd2b6cc admin admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:23.642693 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac552faf-bbd3-46c5-a9fc-287fecd2b6cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:23.646359 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac552faf-bbd3-46c5-a9fc-287fecd2b6cc admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:23.650416 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2169a5-4a44-4726-ab07-0dfbd3bc6d01 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:23.656347 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2232d14b-9bee-48bf-89a1-256fae536c96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76817cd541294331b0023069092e8e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:23.657685 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2232d14b-9bee-48bf-89a1-256fae536c96 service neutron] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:23.658548 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2232d14b-9bee-48bf-89a1-256fae536c96 service neutron] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:23.659707 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2232d14b-9bee-48bf-89a1-256fae536c96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:23.673523 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2232d14b-9bee-48bf-89a1-256fae536c96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:23.675120 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2232d14b-9bee-48bf-89a1-256fae536c96 service neutron] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:23.677591 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 821/1618] 10.222.0.89 () {60 vars in 1144 bytes} [Thu Jul 28 17:44:23 2022] POST /identity/v3/users => generated 297 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:44:23.678962 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cedbdf4-c024-4b99-b2c9-d84f54ed3ac8 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:23.679320 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cedbdf4-c024-4b99-b2c9-d84f54ed3ac8 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:23.680328 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cedbdf4-c024-4b99-b2c9-d84f54ed3ac8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:23.687941 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 798/1619] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:44:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:44:23.689409 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2232d14b-9bee-48bf-89a1-256fae536c96 service neutron] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:23.703216 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2232d14b-9bee-48bf-89a1-256fae536c96 service neutron] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:23.704565 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-6cedbdf4-c024-4b99-b2c9-d84f54ed3ac8 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:44:23.706562 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-6cedbdf4-c024-4b99-b2c9-d84f54ed3ac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:23.710104 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800a841d-0b1e-4d3a-8d1d-0308ab862056 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd10b49b29df94404a71f43273839c252', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:23.710732 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800a841d-0b1e-4d3a-8d1d-0308ab862056 service nova] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:23.711016 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800a841d-0b1e-4d3a-8d1d-0308ab862056 service nova] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:23.711294 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800a841d-0b1e-4d3a-8d1d-0308ab862056 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:23.719406 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 822/1620] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:44:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:44:23.720817 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11e9de9-d3dd-4d28-821a-87561ff629a6 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:23.721100 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11e9de9-d3dd-4d28-821a-87561ff629a6 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:23.721398 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11e9de9-d3dd-4d28-821a-87561ff629a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:23.751279 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:44:23.763551 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-d11e9de9-d3dd-4d28-821a-87561ff629a6 None None] MFA Rules not processed for user `d10b49b29df94404a71f43273839c252`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:44:23.766266 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-d11e9de9-d3dd-4d28-821a-87561ff629a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:23.778073 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 823/1621] 10.222.0.89 () {58 vars in 1059 bytes} [Thu Jul 28 17:44:23 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:44:23.779037 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11e9de9-d3dd-4d28-821a-87561ff629a6 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:23.790947 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11e9de9-d3dd-4d28-821a-87561ff629a6 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:23.797540 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800a841d-0b1e-4d3a-8d1d-0308ab862056 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:23.798983 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a617b0-da0d-40bd-99e6-5892c6f16e8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:23.799464 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800a841d-0b1e-4d3a-8d1d-0308ab862056 service nova] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:23.800271 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a617b0-da0d-40bd-99e6-5892c6f16e8a admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:23.800604 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a617b0-da0d-40bd-99e6-5892c6f16e8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:23.800907 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a617b0-da0d-40bd-99e6-5892c6f16e8a admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:23.801714 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a617b0-da0d-40bd-99e6-5892c6f16e8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:23.803330 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a617b0-da0d-40bd-99e6-5892c6f16e8a admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:23.805032 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:44:23.812302 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 799/1622] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:44:23 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:44:23.820326 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800a841d-0b1e-4d3a-8d1d-0308ab862056 service nova] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:23.829555 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 824/1623] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:44:23 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:44:23.836749 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a617b0-da0d-40bd-99e6-5892c6f16e8a admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:23.840548 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800a841d-0b1e-4d3a-8d1d-0308ab862056 service nova] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:23.846154 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47db9f1-9f10-4586-ad2a-11fa29977250 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76817cd541294331b0023069092e8e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:23.847967 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47db9f1-9f10-4586-ad2a-11fa29977250 service neutron] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:23.849341 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47db9f1-9f10-4586-ad2a-11fa29977250 service neutron] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:23.849986 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47db9f1-9f10-4586-ad2a-11fa29977250 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:23.852179 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a617b0-da0d-40bd-99e6-5892c6f16e8a admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:23.857983 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1103c6-707d-4243-9819-fc7d51c2853f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd10b49b29df94404a71f43273839c252', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:23.858749 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1103c6-707d-4243-9819-fc7d51c2853f service nova] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:23.859228 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1103c6-707d-4243-9819-fc7d51c2853f service nova] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:23.859617 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1103c6-707d-4243-9819-fc7d51c2853f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:23.866130 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47db9f1-9f10-4586-ad2a-11fa29977250 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:23.868319 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47db9f1-9f10-4586-ad2a-11fa29977250 service neutron] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:23.903489 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 800/1624] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:44:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:44:23.906634 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47db9f1-9f10-4586-ad2a-11fa29977250 service neutron] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:23.933521 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47db9f1-9f10-4586-ad2a-11fa29977250 service neutron] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:23.935640 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1103c6-707d-4243-9819-fc7d51c2853f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:23.938112 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1103c6-707d-4243-9819-fc7d51c2853f service nova] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:23.946236 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065318b5-7a75-4a8c-a727-8cb3f9301bf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:23.947190 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065318b5-7a75-4a8c-a727-8cb3f9301bf0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:23.948343 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065318b5-7a75-4a8c-a727-8cb3f9301bf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:23.948720 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065318b5-7a75-4a8c-a727-8cb3f9301bf0 admin admin] PATH_INFO: `/v3/projects/cc0bd418e02f4f4cb1d7ccec3fc7cda9/users/9448c75844b24f5fb671d96d6164fad9/roles/73e457231a824ea98c28d22610be640c` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:23.950230 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 825/1625] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:44:23 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:44:23.970219 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065318b5-7a75-4a8c-a727-8cb3f9301bf0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc0bd418e02f4f4cb1d7ccec3fc7cda9, user_id=9448c75844b24f5fb671d96d6164fad9, role_id=73e457231a824ea98c28d22610be640c)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:23.973723 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065318b5-7a75-4a8c-a727-8cb3f9301bf0 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:23.994416 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 801/1626] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:44:23 2022] PUT /identity/v3/projects/cc0bd418e02f4f4cb1d7ccec3fc7cda9/users/9448c75844b24f5fb671d96d6164fad9/roles/73e457231a824ea98c28d22610be640c => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:44:24.000320 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1103c6-707d-4243-9819-fc7d51c2853f service nova] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:24.017219 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1103c6-707d-4243-9819-fc7d51c2853f service nova] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:24.104548 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d89da8b-e75e-4d41-b480-344d5e8d217f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:24.105772 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d89da8b-e75e-4d41-b480-344d5e8d217f admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:24.106226 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d89da8b-e75e-4d41-b480-344d5e8d217f admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:24.106562 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d89da8b-e75e-4d41-b480-344d5e8d217f admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:24.107382 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d89da8b-e75e-4d41-b480-344d5e8d217f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:24.109969 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d89da8b-e75e-4d41-b480-344d5e8d217f admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:24.117104 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73accce2-1208-47cd-89a2-ee0ed4023c0b None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:24.117576 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73accce2-1208-47cd-89a2-ee0ed4023c0b None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:24.117977 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73accce2-1208-47cd-89a2-ee0ed4023c0b None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:24.119942 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 802/1627] 10.222.0.89 () {54 vars in 912 bytes} [Thu Jul 28 17:44:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:44:24.128972 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889dd925-6165-417d-91a8-7448717a4d4c None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:24.129456 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889dd925-6165-417d-91a8-7448717a4d4c None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:24.129804 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889dd925-6165-417d-91a8-7448717a4d4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:24.147420 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 826/1628] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:44:23 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:44:24.151507 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d89da8b-e75e-4d41-b480-344d5e8d217f admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:24.169322 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d89da8b-e75e-4d41-b480-344d5e8d217f admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:24.175208 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-889dd925-6165-417d-91a8-7448717a4d4c None None] MFA Rules not processed for user `d10b49b29df94404a71f43273839c252`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:44:24.176444 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-889dd925-6165-417d-91a8-7448717a4d4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:24.177022 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa882a1-de76-49fa-aae5-ddc6e4504d36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:24.178339 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa882a1-de76-49fa-aae5-ddc6e4504d36 admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:24.178740 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa882a1-de76-49fa-aae5-ddc6e4504d36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:24.179103 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa882a1-de76-49fa-aae5-ddc6e4504d36 admin admin] PATH_INFO: `/v3/projects/cc0bd418e02f4f4cb1d7ccec3fc7cda9/users/9448c75844b24f5fb671d96d6164fad9/roles/73e457231a824ea98c28d22610be640c` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:24.181815 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa882a1-de76-49fa-aae5-ddc6e4504d36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc0bd418e02f4f4cb1d7ccec3fc7cda9, user_id=9448c75844b24f5fb671d96d6164fad9, role_id=73e457231a824ea98c28d22610be640c)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:24.186595 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa882a1-de76-49fa-aae5-ddc6e4504d36 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:24.206773 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 827/1629] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:44:24 2022] PUT /identity/v3/projects/cc0bd418e02f4f4cb1d7ccec3fc7cda9/users/9448c75844b24f5fb671d96d6164fad9/roles/73e457231a824ea98c28d22610be640c => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:44:24.219720 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f77001b-d840-4939-9de6-51974c31820e None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:24.220016 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f77001b-d840-4939-9de6-51974c31820e None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:24.220766 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f77001b-d840-4939-9de6-51974c31820e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:24.262933 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-8f77001b-d840-4939-9de6-51974c31820e None None] MFA Rules not processed for user `9448c75844b24f5fb671d96d6164fad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:44:24.264185 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-8f77001b-d840-4939-9de6-51974c31820e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:24.319125 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 803/1630] 10.222.0.89 () {58 vars in 1009 bytes} [Thu Jul 28 17:44:24 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:44:24.340870 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889dd925-6165-417d-91a8-7448717a4d4c None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:24.450249 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 828/1631] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:44:24 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:44:24.456117 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9913d22-1928-43b4-88fa-697360a43565 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:24.456412 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9913d22-1928-43b4-88fa-697360a43565 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:24.456691 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9913d22-1928-43b4-88fa-697360a43565 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:24.458574 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 829/1632] 10.222.0.89 () {54 vars in 912 bytes} [Thu Jul 28 17:44:24 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:44:24.460496 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9913d22-1928-43b4-88fa-697360a43565 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:24.470938 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889dd925-6165-417d-91a8-7448717a4d4c None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:24.471201 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:44:24.521010 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:44:24.557278 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c5beb7-9787-40ff-9f8e-9c4699ed0955 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96d512d0c84240918dae6c4a3e8b7b79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:24.558284 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c5beb7-9787-40ff-9f8e-9c4699ed0955 service cinder] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:24.558725 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c5beb7-9787-40ff-9f8e-9c4699ed0955 service cinder] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:24.559158 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c5beb7-9787-40ff-9f8e-9c4699ed0955 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:24.571042 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9913d22-1928-43b4-88fa-697360a43565 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:24.600949 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00f82cc-08da-4724-bac2-bba15b8100b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96d512d0c84240918dae6c4a3e8b7b79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:24.604430 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00f82cc-08da-4724-bac2-bba15b8100b1 service cinder] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:24.604912 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00f82cc-08da-4724-bac2-bba15b8100b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:24.607212 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00f82cc-08da-4724-bac2-bba15b8100b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:24.638305 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00f82cc-08da-4724-bac2-bba15b8100b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:24.643116 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00f82cc-08da-4724-bac2-bba15b8100b1 service cinder] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:24.670160 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 830/1633] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:44:24 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:44:24.671531 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1054d3-f698-46be-8073-dea22343d851 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:24.671819 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1054d3-f698-46be-8073-dea22343d851 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:24.672096 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1054d3-f698-46be-8073-dea22343d851 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:24.731899 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-9d1054d3-f698-46be-8073-dea22343d851 None None] MFA Rules not processed for user `d10b49b29df94404a71f43273839c252`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:44:24.736838 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-9d1054d3-f698-46be-8073-dea22343d851 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:24.758642 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c5beb7-9787-40ff-9f8e-9c4699ed0955 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:24.760208 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c5beb7-9787-40ff-9f8e-9c4699ed0955 service cinder] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:24.772076 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 831/1634] 10.222.0.89 () {58 vars in 1009 bytes} [Thu Jul 28 17:44:24 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:44:24.797257 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 804/1635] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:44:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 17:44:24.859722 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1054d3-f698-46be-8073-dea22343d851 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:24.876585 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1054d3-f698-46be-8073-dea22343d851 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:24.887995 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cf443a-796c-4310-8101-cfd985a52a9b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96d512d0c84240918dae6c4a3e8b7b79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:24.889005 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cf443a-796c-4310-8101-cfd985a52a9b service cinder] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:24.889525 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cf443a-796c-4310-8101-cfd985a52a9b service cinder] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:24.890913 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cf443a-796c-4310-8101-cfd985a52a9b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:24.919039 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cf443a-796c-4310-8101-cfd985a52a9b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:24.920653 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cf443a-796c-4310-8101-cfd985a52a9b service cinder] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:24.950331 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 832/1636] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:44:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:44:25.105426 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c5beb7-9787-40ff-9f8e-9c4699ed0955 service cinder] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:25.116007 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c5beb7-9787-40ff-9f8e-9c4699ed0955 service cinder] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:25.122209 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d69a77-67e5-4d81-aaf8-a797fddf0794 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96d512d0c84240918dae6c4a3e8b7b79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:25.123100 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d69a77-67e5-4d81-aaf8-a797fddf0794 service cinder] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:25.123598 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d69a77-67e5-4d81-aaf8-a797fddf0794 service cinder] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:25.124115 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d69a77-67e5-4d81-aaf8-a797fddf0794 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:25.211246 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d69a77-67e5-4d81-aaf8-a797fddf0794 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:25.214149 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d69a77-67e5-4d81-aaf8-a797fddf0794 service cinder] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:25.228305 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 805/1637] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:44:25 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:44:25.655171 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc7b3c9-1d66-42f3-834c-b6a8d8d57048 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:25.656011 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc7b3c9-1d66-42f3-834c-b6a8d8d57048 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:25.656933 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc7b3c9-1d66-42f3-834c-b6a8d8d57048 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:25.715243 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-6bc7b3c9-1d66-42f3-834c-b6a8d8d57048 None None] MFA Rules not processed for user `4f2ffc8d7e9044d2929416598eee79f6`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:44:25.716399 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-6bc7b3c9-1d66-42f3-834c-b6a8d8d57048 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:25.861463 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d69a77-67e5-4d81-aaf8-a797fddf0794 service cinder] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:25.886398 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 833/1638] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:44:25 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:44:25.891412 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc7b3c9-1d66-42f3-834c-b6a8d8d57048 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:25.964199 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d69a77-67e5-4d81-aaf8-a797fddf0794 service cinder] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:25.971176 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28321c74-49f9-4289-bc7e-556bbf268e71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:25.971176 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28321c74-49f9-4289-bc7e-556bbf268e71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:25.971453 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28321c74-49f9-4289-bc7e-556bbf268e71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:25.971453 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28321c74-49f9-4289-bc7e-556bbf268e71 admin admin] PATH_INFO: `/v3/users/0d61a50992b241efbad5af3ece48a8ea` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:25.972667 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28321c74-49f9-4289-bc7e-556bbf268e71 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d61a50992b241efbad5af3ece48a8ea)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:25.973789 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: WARNING py.warnings [None req-28321c74-49f9-4289-bc7e-556bbf268e71 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 28 17:44:25.973789 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: warnings.warn(msg) Jul 28 17:44:25.973789 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: Jul 28 17:44:25.975050 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28321c74-49f9-4289-bc7e-556bbf268e71 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:25.987317 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc7b3c9-1d66-42f3-834c-b6a8d8d57048 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:25.993539 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d652f45-51b6-47fc-be64-5ab163380239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:25.994680 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d652f45-51b6-47fc-be64-5ab163380239 admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:25.994942 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d652f45-51b6-47fc-be64-5ab163380239 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:25.995218 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d652f45-51b6-47fc-be64-5ab163380239 admin admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:25.995890 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d652f45-51b6-47fc-be64-5ab163380239 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:25.998083 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d652f45-51b6-47fc-be64-5ab163380239 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:26.003620 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:44:26.038519 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:44:26.056868 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 834/1639] 10.222.0.89 () {60 vars in 1153 bytes} [Thu Jul 28 17:44:25 2022] POST /identity/v3/projects => generated 405 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:44:26.058280 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d652f45-51b6-47fc-be64-5ab163380239 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:26.080842 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-28321c74-49f9-4289-bc7e-556bbf268e71 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d61a50992b241efbad5af3ece48a8ea'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:44:26.099977 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-28321c74-49f9-4289-bc7e-556bbf268e71 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d61a50992b241efbad5af3ece48a8ea'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:44:26.114846 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-28321c74-49f9-4289-bc7e-556bbf268e71 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d61a50992b241efbad5af3ece48a8ea'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:44:26.116035 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-28321c74-49f9-4289-bc7e-556bbf268e71 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d61a50992b241efbad5af3ece48a8ea'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:44:26.116806 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-28321c74-49f9-4289-bc7e-556bbf268e71 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d61a50992b241efbad5af3ece48a8ea'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:44:26.160816 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 806/1640] 10.222.0.89 () {58 vars in 1224 bytes} [Thu Jul 28 17:44:25 2022] DELETE /identity/v3/users/0d61a50992b241efbad5af3ece48a8ea => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:44:26.166470 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0babf5d-349f-4598-ace0-2b5c64b97fe1 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:26.166589 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0babf5d-349f-4598-ace0-2b5c64b97fe1 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:26.166867 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0babf5d-349f-4598-ace0-2b5c64b97fe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:26.191829 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d652f45-51b6-47fc-be64-5ab163380239 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:26.208227 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-a0babf5d-349f-4598-ace0-2b5c64b97fe1 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:44:26.209479 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-a0babf5d-349f-4598-ace0-2b5c64b97fe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:26.305363 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 807/1641] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:44:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 17:44:26.316648 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e942a1-ce79-49ff-b3be-2284a10a2a1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:26.317321 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e942a1-ce79-49ff-b3be-2284a10a2a1a admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:26.317601 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e942a1-ce79-49ff-b3be-2284a10a2a1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:26.317871 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e942a1-ce79-49ff-b3be-2284a10a2a1a admin admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:26.319050 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e942a1-ce79-49ff-b3be-2284a10a2a1a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:26.320098 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0babf5d-349f-4598-ace0-2b5c64b97fe1 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:26.321990 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-a0babf5d-349f-4598-ace0-2b5c64b97fe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:26.322110 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e942a1-ce79-49ff-b3be-2284a10a2a1a admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:26.354728 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 835/1642] 10.222.0.89 () {60 vars in 1144 bytes} [Thu Jul 28 17:44:26 2022] POST /identity/v3/users => generated 309 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:44:26.358366 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e942a1-ce79-49ff-b3be-2284a10a2a1a admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:26.361594 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-87e942a1-ce79-49ff-b3be-2284a10a2a1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:26.380668 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e942a1-ce79-49ff-b3be-2284a10a2a1a admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:26.386320 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed16c4c4-c653-4ce9-87f0-95367a8fe1cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:26.388772 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed16c4c4-c653-4ce9-87f0-95367a8fe1cb admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:26.389422 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed16c4c4-c653-4ce9-87f0-95367a8fe1cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:26.390006 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed16c4c4-c653-4ce9-87f0-95367a8fe1cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:26.391822 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed16c4c4-c653-4ce9-87f0-95367a8fe1cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:26.395145 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed16c4c4-c653-4ce9-87f0-95367a8fe1cb admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:26.408885 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 836/1643] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:44:26 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:44:26.412751 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed16c4c4-c653-4ce9-87f0-95367a8fe1cb admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:26.424773 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0babf5d-349f-4598-ace0-2b5c64b97fe1 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:26.426756 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed16c4c4-c653-4ce9-87f0-95367a8fe1cb admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:26.432173 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0899a108-0d9b-4114-81b9-3afcb96b9bf4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76817cd541294331b0023069092e8e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:26.433595 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0899a108-0d9b-4114-81b9-3afcb96b9bf4 service neutron] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:26.435035 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0899a108-0d9b-4114-81b9-3afcb96b9bf4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:26.436436 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0899a108-0d9b-4114-81b9-3afcb96b9bf4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:26.438859 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435803b7-30f7-47bf-9dea-f67501ed28ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:26.440127 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435803b7-30f7-47bf-9dea-f67501ed28ec admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:26.442009 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435803b7-30f7-47bf-9dea-f67501ed28ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:26.443381 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435803b7-30f7-47bf-9dea-f67501ed28ec admin admin] PATH_INFO: `/v3/projects/414a3cb8d5af4b038fa928047d55e664/users/c4b0ff3bf2874b1abbc877f64475ba45/roles/73e457231a824ea98c28d22610be640c` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:26.451013 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0899a108-0d9b-4114-81b9-3afcb96b9bf4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:26.453829 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0899a108-0d9b-4114-81b9-3afcb96b9bf4 service neutron] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:26.466000 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 808/1644] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:44:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:44:26.476804 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435803b7-30f7-47bf-9dea-f67501ed28ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=414a3cb8d5af4b038fa928047d55e664, user_id=c4b0ff3bf2874b1abbc877f64475ba45, role_id=73e457231a824ea98c28d22610be640c)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:26.480218 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435803b7-30f7-47bf-9dea-f67501ed28ec admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:26.504760 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 837/1645] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:44:26 2022] PUT /identity/v3/projects/414a3cb8d5af4b038fa928047d55e664/users/c4b0ff3bf2874b1abbc877f64475ba45/roles/73e457231a824ea98c28d22610be640c => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 17:44:26.508951 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0899a108-0d9b-4114-81b9-3afcb96b9bf4 service neutron] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:26.523378 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0899a108-0d9b-4114-81b9-3afcb96b9bf4 service neutron] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:26.636358 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:44:26.709674 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e9ee40-5a74-4f15-a6df-2548439550d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:26.710664 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e9ee40-5a74-4f15-a6df-2548439550d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:26.712401 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e9ee40-5a74-4f15-a6df-2548439550d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:26.712787 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e9ee40-5a74-4f15-a6df-2548439550d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:26.715006 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e9ee40-5a74-4f15-a6df-2548439550d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:26.720923 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e9ee40-5a74-4f15-a6df-2548439550d5 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:26.750696 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 809/1646] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:44:26 2022] GET /identity/v3/roles => generated 1420 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:44:26.751438 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435803b7-30f7-47bf-9dea-f67501ed28ec admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:26.776194 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435803b7-30f7-47bf-9dea-f67501ed28ec admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:26.801903 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076f1a7a-f102-4769-8e60-331873f1e789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:26.802663 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076f1a7a-f102-4769-8e60-331873f1e789 admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:26.803001 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076f1a7a-f102-4769-8e60-331873f1e789 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:26.803281 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076f1a7a-f102-4769-8e60-331873f1e789 admin admin] PATH_INFO: `/v3/projects/414a3cb8d5af4b038fa928047d55e664/users/c4b0ff3bf2874b1abbc877f64475ba45/roles/73e457231a824ea98c28d22610be640c` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:26.806690 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076f1a7a-f102-4769-8e60-331873f1e789 admin admin] RBAC: Authorizing `identity:create_grant(project_id=414a3cb8d5af4b038fa928047d55e664, user_id=c4b0ff3bf2874b1abbc877f64475ba45, role_id=73e457231a824ea98c28d22610be640c)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:26.810217 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076f1a7a-f102-4769-8e60-331873f1e789 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:26.831568 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 838/1647] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:44:26 2022] PUT /identity/v3/projects/414a3cb8d5af4b038fa928047d55e664/users/c4b0ff3bf2874b1abbc877f64475ba45/roles/73e457231a824ea98c28d22610be640c => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 17:44:26.934393 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e9ee40-5a74-4f15-a6df-2548439550d5 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:26.940799 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-d6e9ee40-5a74-4f15-a6df-2548439550d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:26.977804 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e9ee40-5a74-4f15-a6df-2548439550d5 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:27.150701 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05eea1ba-4725-42c8-9036-76e0c87865be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:27.153333 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05eea1ba-4725-42c8-9036-76e0c87865be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:27.156739 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05eea1ba-4725-42c8-9036-76e0c87865be admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:27.157859 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05eea1ba-4725-42c8-9036-76e0c87865be admin admin] PATH_INFO: `/v3/projects/29887ef23f954dc18aea8bd7e847cabc` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:27.168092 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05eea1ba-4725-42c8-9036-76e0c87865be admin admin] RBAC: Authorizing `identity:delete_project(project_id=29887ef23f954dc18aea8bd7e847cabc)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:27.175569 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05eea1ba-4725-42c8-9036-76e0c87865be admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:27.307534 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-05eea1ba-4725-42c8-9036-76e0c87865be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29887ef23f954dc18aea8bd7e847cabc', 'request_id': 'req-05eea1ba-4725-42c8-9036-76e0c87865be'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:44:27.391605 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-05eea1ba-4725-42c8-9036-76e0c87865be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29887ef23f954dc18aea8bd7e847cabc', 'request_id': 'req-05eea1ba-4725-42c8-9036-76e0c87865be'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:44:27.391605 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-05eea1ba-4725-42c8-9036-76e0c87865be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29887ef23f954dc18aea8bd7e847cabc', 'request_id': 'req-05eea1ba-4725-42c8-9036-76e0c87865be'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:44:27.394305 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-05eea1ba-4725-42c8-9036-76e0c87865be admin admin] The token cache is being invalidate because project 29887ef23f954dc18aea8bd7e847cabc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80325) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 17:44:27.394748 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-05eea1ba-4725-42c8-9036-76e0c87865be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:44:27.427436 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-05eea1ba-4725-42c8-9036-76e0c87865be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:44:27.427436 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 810/1648] 10.222.0.89 () {58 vars in 1233 bytes} [Thu Jul 28 17:44:26 2022] DELETE /identity/v3/projects/29887ef23f954dc18aea8bd7e847cabc => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:44:27.429092 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199741c0-ce52-44e8-b3f8-59d55200b6df None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:27.429092 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199741c0-ce52-44e8-b3f8-59d55200b6df None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:27.429092 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199741c0-ce52-44e8-b3f8-59d55200b6df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:27.466462 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-199741c0-ce52-44e8-b3f8-59d55200b6df None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:44:27.468633 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-199741c0-ce52-44e8-b3f8-59d55200b6df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:27.574385 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:44:27.618990 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 839/1649] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:44:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:44:27.628265 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2163aa49-5efe-401a-86ff-e22db51f15c8 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:27.629646 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2163aa49-5efe-401a-86ff-e22db51f15c8 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:27.630215 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2163aa49-5efe-401a-86ff-e22db51f15c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:27.692080 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-2163aa49-5efe-401a-86ff-e22db51f15c8 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:44:27.696379 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-2163aa49-5efe-401a-86ff-e22db51f15c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:27.730113 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 811/1650] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:44:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:44:27.735944 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199741c0-ce52-44e8-b3f8-59d55200b6df None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:27.747518 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199741c0-ce52-44e8-b3f8-59d55200b6df None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:27.763407 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ecc845-838c-477d-afa3-5139f24d4bdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:27.764133 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ecc845-838c-477d-afa3-5139f24d4bdc admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:27.764680 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ecc845-838c-477d-afa3-5139f24d4bdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:27.765655 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ecc845-838c-477d-afa3-5139f24d4bdc admin admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:27.766035 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ecc845-838c-477d-afa3-5139f24d4bdc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:44:27.766339 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ecc845-838c-477d-afa3-5139f24d4bdc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:27.767961 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ecc845-838c-477d-afa3-5139f24d4bdc admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:27.798090 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 840/1651] 10.222.0.89 () {58 vars in 1153 bytes} [Thu Jul 28 17:44:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:44:27.802227 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2163aa49-5efe-401a-86ff-e22db51f15c8 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:27.822603 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2163aa49-5efe-401a-86ff-e22db51f15c8 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:27.839866 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2036e97d-fdfb-4d53-bc3a-c741b843bce2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:27.840165 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2036e97d-fdfb-4d53-bc3a-c741b843bce2 admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:27.840460 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2036e97d-fdfb-4d53-bc3a-c741b843bce2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:27.840741 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2036e97d-fdfb-4d53-bc3a-c741b843bce2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:27.844345 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2036e97d-fdfb-4d53-bc3a-c741b843bce2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:27.848462 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2036e97d-fdfb-4d53-bc3a-c741b843bce2 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:27.920802 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 812/1652] 10.222.0.89 () {60 vars in 1153 bytes} [Thu Jul 28 17:44:27 2022] POST /identity/v3/projects => generated 367 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:44:27.923017 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ecc845-838c-477d-afa3-5139f24d4bdc admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:27.943922 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ecc845-838c-477d-afa3-5139f24d4bdc admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:28.096972 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2058de-f1ab-4582-821b-22086d6dcaed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:28.099309 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2058de-f1ab-4582-821b-22086d6dcaed admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:28.100905 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2058de-f1ab-4582-821b-22086d6dcaed admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:28.101459 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2058de-f1ab-4582-821b-22086d6dcaed admin admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:28.103930 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2058de-f1ab-4582-821b-22086d6dcaed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:28.107012 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2058de-f1ab-4582-821b-22086d6dcaed admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:28.162977 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 841/1653] 10.222.0.89 () {60 vars in 1144 bytes} [Thu Jul 28 17:44:27 2022] POST /identity/v3/users => generated 284 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:44:28.166739 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2036e97d-fdfb-4d53-bc3a-c741b843bce2 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:28.201209 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2036e97d-fdfb-4d53-bc3a-c741b843bce2 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:28.219811 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df4662b-3c2e-4554-a646-434aee4d2b28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:28.221965 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df4662b-3c2e-4554-a646-434aee4d2b28 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:28.222712 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df4662b-3c2e-4554-a646-434aee4d2b28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:28.223652 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df4662b-3c2e-4554-a646-434aee4d2b28 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:28.225598 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df4662b-3c2e-4554-a646-434aee4d2b28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:28.227268 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df4662b-3c2e-4554-a646-434aee4d2b28 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:28.261061 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 813/1654] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:44:28 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:44:28.265543 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2058de-f1ab-4582-821b-22086d6dcaed admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:28.280700 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2058de-f1ab-4582-821b-22086d6dcaed admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:28.291721 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8497bd-f39a-452b-b5c6-be4fedd9b68d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:28.293598 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8497bd-f39a-452b-b5c6-be4fedd9b68d admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:28.293888 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8497bd-f39a-452b-b5c6-be4fedd9b68d admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:28.294171 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8497bd-f39a-452b-b5c6-be4fedd9b68d admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:28.297569 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8497bd-f39a-452b-b5c6-be4fedd9b68d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:28.299332 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e11cef-de4c-4e0d-bdfb-ad6d9cbcbb6f None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:28.299565 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8497bd-f39a-452b-b5c6-be4fedd9b68d admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:28.299661 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e11cef-de4c-4e0d-bdfb-ad6d9cbcbb6f None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:28.299888 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e11cef-de4c-4e0d-bdfb-ad6d9cbcbb6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:28.331359 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 842/1655] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:44:28 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:44:28.332339 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8497bd-f39a-452b-b5c6-be4fedd9b68d admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:28.355211 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-c0e11cef-de4c-4e0d-bdfb-ad6d9cbcbb6f None None] MFA Rules not processed for user `c4b0ff3bf2874b1abbc877f64475ba45`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:44:28.356473 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-c0e11cef-de4c-4e0d-bdfb-ad6d9cbcbb6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:28.380555 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8497bd-f39a-452b-b5c6-be4fedd9b68d admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:28.388790 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f95f51-d472-41f4-9916-a8bf74573596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:28.390019 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f95f51-d472-41f4-9916-a8bf74573596 admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:28.390423 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f95f51-d472-41f4-9916-a8bf74573596 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:28.390781 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f95f51-d472-41f4-9916-a8bf74573596 admin admin] PATH_INFO: `/v3/projects/c200bf1255164cc5aa6a81157a1eb894/users/aeb828c53f8444c2968c1542b3962ea0/roles/11f8731756fe4caca13efa61b71b11c5` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:28.412777 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:44:28.419856 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f95f51-d472-41f4-9916-a8bf74573596 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c200bf1255164cc5aa6a81157a1eb894, user_id=aeb828c53f8444c2968c1542b3962ea0, role_id=11f8731756fe4caca13efa61b71b11c5)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:28.425545 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f95f51-d472-41f4-9916-a8bf74573596 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:28.459745 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 843/1656] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:44:28 2022] PUT /identity/v3/projects/c200bf1255164cc5aa6a81157a1eb894/users/aeb828c53f8444c2968c1542b3962ea0/roles/11f8731756fe4caca13efa61b71b11c5 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:44:28.481196 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4144b903-5930-43ed-aba2-c9ceaac19ad2 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:28.481999 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4144b903-5930-43ed-aba2-c9ceaac19ad2 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:28.483125 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4144b903-5930-43ed-aba2-c9ceaac19ad2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:28.529026 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-4144b903-5930-43ed-aba2-c9ceaac19ad2 None None] MFA Rules not processed for user `aeb828c53f8444c2968c1542b3962ea0`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:44:28.532594 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-4144b903-5930-43ed-aba2-c9ceaac19ad2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:28.627437 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 844/1657] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:44:28 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 17:44:28.635779 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4144b903-5930-43ed-aba2-c9ceaac19ad2 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:28.653555 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 814/1658] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:44:28 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 355 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:44:28.698546 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:44:28.832269 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4144b903-5930-43ed-aba2-c9ceaac19ad2 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:28.869027 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfd7133-7e53-4ffb-9c8f-f45cdf362a89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:28.888494 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfd7133-7e53-4ffb-9c8f-f45cdf362a89 admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:28.889121 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfd7133-7e53-4ffb-9c8f-f45cdf362a89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:28.889563 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfd7133-7e53-4ffb-9c8f-f45cdf362a89 admin admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:28.891339 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfd7133-7e53-4ffb-9c8f-f45cdf362a89 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:28.899889 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfd7133-7e53-4ffb-9c8f-f45cdf362a89 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:28.995212 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 845/1659] 10.222.0.89 () {60 vars in 1153 bytes} [Thu Jul 28 17:44:28 2022] POST /identity/v3/projects => generated 367 bytes in 362 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:44:28.998390 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e11cef-de4c-4e0d-bdfb-ad6d9cbcbb6f None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:29.021946 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e11cef-de4c-4e0d-bdfb-ad6d9cbcbb6f None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:29.280165 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0c2a4e-a03f-4747-9510-64188ecd12ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:29.281461 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0c2a4e-a03f-4747-9510-64188ecd12ca admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:29.281708 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0c2a4e-a03f-4747-9510-64188ecd12ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:29.281976 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0c2a4e-a03f-4747-9510-64188ecd12ca admin admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:29.282656 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0c2a4e-a03f-4747-9510-64188ecd12ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:29.284578 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0c2a4e-a03f-4747-9510-64188ecd12ca admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:29.342514 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 815/1660] 10.222.0.89 () {60 vars in 1144 bytes} [Thu Jul 28 17:44:28 2022] POST /identity/v3/users => generated 290 bytes in 345 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:44:29.347824 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfd7133-7e53-4ffb-9c8f-f45cdf362a89 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:29.372673 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfd7133-7e53-4ffb-9c8f-f45cdf362a89 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:29.380161 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2944770-0f12-48dd-855a-b943c04ec1a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:29.381319 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2944770-0f12-48dd-855a-b943c04ec1a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:29.381819 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2944770-0f12-48dd-855a-b943c04ec1a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:29.382296 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2944770-0f12-48dd-855a-b943c04ec1a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:29.383751 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2944770-0f12-48dd-855a-b943c04ec1a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:29.391257 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2944770-0f12-48dd-855a-b943c04ec1a0 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:29.408628 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 846/1661] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:44:29 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:44:29.412590 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0c2a4e-a03f-4747-9510-64188ecd12ca admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:29.426692 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0c2a4e-a03f-4747-9510-64188ecd12ca admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:29.444547 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67867ec8-477a-45b8-8a36-8d8770f6feb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:29.445389 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67867ec8-477a-45b8-8a36-8d8770f6feb4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:29.445749 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67867ec8-477a-45b8-8a36-8d8770f6feb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:29.446156 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67867ec8-477a-45b8-8a36-8d8770f6feb4 admin admin] PATH_INFO: `/v3/projects/db52bab56abd4dd98539858e1c5eefa8/users/005c03fdb4584f4ba7764d7246bfc5ea/roles/73e457231a824ea98c28d22610be640c` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:29.478508 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67867ec8-477a-45b8-8a36-8d8770f6feb4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db52bab56abd4dd98539858e1c5eefa8, user_id=005c03fdb4584f4ba7764d7246bfc5ea, role_id=73e457231a824ea98c28d22610be640c)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:29.481280 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67867ec8-477a-45b8-8a36-8d8770f6feb4 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:29.540102 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 816/1662] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:44:29 2022] PUT /identity/v3/projects/db52bab56abd4dd98539858e1c5eefa8/users/005c03fdb4584f4ba7764d7246bfc5ea/roles/73e457231a824ea98c28d22610be640c => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:44:29.543969 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2944770-0f12-48dd-855a-b943c04ec1a0 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:29.565762 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2944770-0f12-48dd-855a-b943c04ec1a0 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:29.762342 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb53c636-2761-407f-9cf2-809722e32439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:29.763000 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb53c636-2761-407f-9cf2-809722e32439 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:29.763286 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb53c636-2761-407f-9cf2-809722e32439 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:29.763563 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb53c636-2761-407f-9cf2-809722e32439 admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:29.764283 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb53c636-2761-407f-9cf2-809722e32439 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:29.766302 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb53c636-2761-407f-9cf2-809722e32439 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:29.784680 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 847/1663] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:44:29 2022] GET /identity/v3/roles => generated 1420 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:44:29.789240 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67867ec8-477a-45b8-8a36-8d8770f6feb4 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:29.821307 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67867ec8-477a-45b8-8a36-8d8770f6feb4 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:29.836055 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94e5352-c24d-400a-ba7f-6c0f69ecd9ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:29.836764 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94e5352-c24d-400a-ba7f-6c0f69ecd9ad admin admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:29.837047 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94e5352-c24d-400a-ba7f-6c0f69ecd9ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:29.837401 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94e5352-c24d-400a-ba7f-6c0f69ecd9ad admin admin] PATH_INFO: `/v3/projects/db52bab56abd4dd98539858e1c5eefa8/users/005c03fdb4584f4ba7764d7246bfc5ea/roles/73e457231a824ea98c28d22610be640c` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:29.840169 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94e5352-c24d-400a-ba7f-6c0f69ecd9ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=db52bab56abd4dd98539858e1c5eefa8, user_id=005c03fdb4584f4ba7764d7246bfc5ea, role_id=73e457231a824ea98c28d22610be640c)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:29.843425 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94e5352-c24d-400a-ba7f-6c0f69ecd9ad admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:29.875310 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 817/1664] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:44:29 2022] PUT /identity/v3/projects/db52bab56abd4dd98539858e1c5eefa8/users/005c03fdb4584f4ba7764d7246bfc5ea/roles/73e457231a824ea98c28d22610be640c => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 17:44:29.877115 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e7da96-f649-4684-991a-51a857dd0dec None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:29.877974 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e7da96-f649-4684-991a-51a857dd0dec None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:29.878724 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e7da96-f649-4684-991a-51a857dd0dec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:29.921386 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-b4e7da96-f649-4684-991a-51a857dd0dec None None] MFA Rules not processed for user `005c03fdb4584f4ba7764d7246bfc5ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:44:29.923060 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-b4e7da96-f649-4684-991a-51a857dd0dec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:30.035466 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 848/1665] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:44:29 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:44:30.050770 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94e5352-c24d-400a-ba7f-6c0f69ecd9ad admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:30.052950 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-b94e5352-c24d-400a-ba7f-6c0f69ecd9ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:30.136573 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94e5352-c24d-400a-ba7f-6c0f69ecd9ad admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:30.140954 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d650b8a2-af8c-46a6-8afd-8c12f074c131 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96d512d0c84240918dae6c4a3e8b7b79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:30.141633 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d650b8a2-af8c-46a6-8afd-8c12f074c131 service cinder] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:30.141915 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d650b8a2-af8c-46a6-8afd-8c12f074c131 service cinder] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:30.142186 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d650b8a2-af8c-46a6-8afd-8c12f074c131 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:30.153055 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d650b8a2-af8c-46a6-8afd-8c12f074c131 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:30.155071 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d650b8a2-af8c-46a6-8afd-8c12f074c131 service cinder] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:30.169879 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 818/1666] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:44:30 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:44:30.274130 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e7da96-f649-4684-991a-51a857dd0dec None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:30.275851 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-b4e7da96-f649-4684-991a-51a857dd0dec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:30.310806 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e7da96-f649-4684-991a-51a857dd0dec None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:30.318429 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6c0d6c-73c5-4dcd-8d62-812b1644c04b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96d512d0c84240918dae6c4a3e8b7b79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:30.319068 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6c0d6c-73c5-4dcd-8d62-812b1644c04b service cinder] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:30.319345 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6c0d6c-73c5-4dcd-8d62-812b1644c04b service cinder] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:30.319617 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6c0d6c-73c5-4dcd-8d62-812b1644c04b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:30.349141 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:44:30.455748 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6c0d6c-73c5-4dcd-8d62-812b1644c04b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:30.460066 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6c0d6c-73c5-4dcd-8d62-812b1644c04b service cinder] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:30.492850 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 849/1667] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:44:30 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:44:31.317337 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aa540a-9f0f-454a-af08-d84480950e3f None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:31.317805 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aa540a-9f0f-454a-af08-d84480950e3f None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:31.317887 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aa540a-9f0f-454a-af08-d84480950e3f None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:31.319254 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 819/1668] 10.222.0.89 () {54 vars in 921 bytes} [Thu Jul 28 17:44:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:44:31.375458 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1079ba-2ab6-4581-b83a-042dc2d3af87 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:31.377670 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1079ba-2ab6-4581-b83a-042dc2d3af87 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:31.378548 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1079ba-2ab6-4581-b83a-042dc2d3af87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:31.414588 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-7c1079ba-2ab6-4581-b83a-042dc2d3af87 None None] MFA Rules not processed for user `e4bd290018394b0d955cd376ce35b77f`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:44:31.416860 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-7c1079ba-2ab6-4581-b83a-042dc2d3af87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:31.566702 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 850/1669] 10.222.0.89 () {58 vars in 1018 bytes} [Thu Jul 28 17:44:31 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:44:31.614213 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86c6e0e-e0f7-4e8f-a134-734ceb9d242c None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:31.614761 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86c6e0e-e0f7-4e8f-a134-734ceb9d242c None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:31.615186 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86c6e0e-e0f7-4e8f-a134-734ceb9d242c None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:31.616701 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 820/1670] 10.222.0.89 () {54 vars in 961 bytes} [Thu Jul 28 17:44:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:44:31.637055 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441a9442-b0a2-4656-a243-84a14ba43acc None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:31.637384 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441a9442-b0a2-4656-a243-84a14ba43acc None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:31.637664 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441a9442-b0a2-4656-a243-84a14ba43acc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:31.680848 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-441a9442-b0a2-4656-a243-84a14ba43acc None None] MFA Rules not processed for user `e4bd290018394b0d955cd376ce35b77f`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:44:31.682717 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-441a9442-b0a2-4656-a243-84a14ba43acc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:31.707748 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 851/1671] 10.222.0.89 () {58 vars in 1058 bytes} [Thu Jul 28 17:44:31 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:44:31.719464 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86c6e0e-e0f7-4e8f-a134-734ceb9d242c None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:31.731853 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:44:31.747106 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86c6e0e-e0f7-4e8f-a134-734ceb9d242c None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:31.758852 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1cb171-a90d-40c5-9787-14aa45fdc0bb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4bd290018394b0d955cd376ce35b77f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:31.759632 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1cb171-a90d-40c5-9787-14aa45fdc0bb service placement] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:31.759976 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1cb171-a90d-40c5-9787-14aa45fdc0bb service placement] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:31.760328 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1cb171-a90d-40c5-9787-14aa45fdc0bb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:31.781309 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1cb171-a90d-40c5-9787-14aa45fdc0bb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:31.783042 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1cb171-a90d-40c5-9787-14aa45fdc0bb service placement] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:31.811402 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 821/1672] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:44:31 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:44:35.254193 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441a9442-b0a2-4656-a243-84a14ba43acc None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:35.408643 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441a9442-b0a2-4656-a243-84a14ba43acc None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:35.419343 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9664dc1-719d-4990-94c2-bc961d2ee639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:35.420000 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9664dc1-719d-4990-94c2-bc961d2ee639 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:35.420277 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9664dc1-719d-4990-94c2-bc961d2ee639 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:35.420548 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9664dc1-719d-4990-94c2-bc961d2ee639 admin admin] PATH_INFO: `/v3/users/aeb828c53f8444c2968c1542b3962ea0` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:35.422055 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9664dc1-719d-4990-94c2-bc961d2ee639 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aeb828c53f8444c2968c1542b3962ea0)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:35.425681 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9664dc1-719d-4990-94c2-bc961d2ee639 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:35.532844 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1531b687-ebe5-40a2-ab5d-8c6a5b356ac8 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:35.533425 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1531b687-ebe5-40a2-ab5d-8c6a5b356ac8 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:35.533562 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1531b687-ebe5-40a2-ab5d-8c6a5b356ac8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:35.543787 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:44:35.547805 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-b9664dc1-719d-4990-94c2-bc961d2ee639 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aeb828c53f8444c2968c1542b3962ea0'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:44:35.564345 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-b9664dc1-719d-4990-94c2-bc961d2ee639 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aeb828c53f8444c2968c1542b3962ea0'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:44:35.565328 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-b9664dc1-719d-4990-94c2-bc961d2ee639 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aeb828c53f8444c2968c1542b3962ea0'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:44:35.579259 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-b9664dc1-719d-4990-94c2-bc961d2ee639 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aeb828c53f8444c2968c1542b3962ea0'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:44:35.595810 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-1531b687-ebe5-40a2-ab5d-8c6a5b356ac8 None None] MFA Rules not processed for user `14327b8153ab4de782d4bcfef7681fc1`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:44:35.597426 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-1531b687-ebe5-40a2-ab5d-8c6a5b356ac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:35.598948 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-b9664dc1-719d-4990-94c2-bc961d2ee639 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aeb828c53f8444c2968c1542b3962ea0'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:44:35.617188 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 852/1673] 10.222.0.89 () {58 vars in 1224 bytes} [Thu Jul 28 17:44:35 2022] DELETE /identity/v3/users/aeb828c53f8444c2968c1542b3962ea0 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:44:35.623098 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ae0b78-f4a6-4470-9953-e9de366090a0 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:35.625487 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ae0b78-f4a6-4470-9953-e9de366090a0 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:35.625940 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ae0b78-f4a6-4470-9953-e9de366090a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:35.679965 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-47ae0b78-f4a6-4470-9953-e9de366090a0 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:44:35.684401 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-47ae0b78-f4a6-4470-9953-e9de366090a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:35.788393 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 822/1674] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:44:35 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:44:35.802668 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74209dc-59e0-4123-90dd-dab1c9232ff4 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:35.802964 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74209dc-59e0-4123-90dd-dab1c9232ff4 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:35.803248 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74209dc-59e0-4123-90dd-dab1c9232ff4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:35.818788 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 853/1675] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:44:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:44:35.837758 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ae0b78-f4a6-4470-9953-e9de366090a0 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:35.839400 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-47ae0b78-f4a6-4470-9953-e9de366090a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:35.841773 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-c74209dc-59e0-4123-90dd-dab1c9232ff4 None None] MFA Rules not processed for user `14327b8153ab4de782d4bcfef7681fc1`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:44:35.843608 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-c74209dc-59e0-4123-90dd-dab1c9232ff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:35.994634 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 823/1676] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:44:35 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:44:35.995637 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74209dc-59e0-4123-90dd-dab1c9232ff4 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:36.001404 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-c74209dc-59e0-4123-90dd-dab1c9232ff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:36.035358 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ae0b78-f4a6-4470-9953-e9de366090a0 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:36.049877 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3f0166-1e09-41a9-909e-5aa37cbed9ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76817cd541294331b0023069092e8e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:36.050517 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3f0166-1e09-41a9-909e-5aa37cbed9ec service neutron] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:36.052159 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3f0166-1e09-41a9-909e-5aa37cbed9ec service neutron] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:36.052292 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3f0166-1e09-41a9-909e-5aa37cbed9ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:36.076026 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3f0166-1e09-41a9-909e-5aa37cbed9ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:36.081595 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3f0166-1e09-41a9-909e-5aa37cbed9ec service neutron] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:36.112110 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 854/1677] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:44:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 17:44:36.113141 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3f0166-1e09-41a9-909e-5aa37cbed9ec service neutron] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:36.119748 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-6c3f0166-1e09-41a9-909e-5aa37cbed9ec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:36.151177 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3f0166-1e09-41a9-909e-5aa37cbed9ec service neutron] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:36.163508 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fbd372-120c-4426-afa2-ca0dfc6945ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:36.165035 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fbd372-120c-4426-afa2-ca0dfc6945ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:36.165344 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fbd372-120c-4426-afa2-ca0dfc6945ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:36.165623 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fbd372-120c-4426-afa2-ca0dfc6945ad admin admin] PATH_INFO: `/v3/users/4f2ffc8d7e9044d2929416598eee79f6` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:36.168072 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fbd372-120c-4426-afa2-ca0dfc6945ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f2ffc8d7e9044d2929416598eee79f6)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:36.170678 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74209dc-59e0-4123-90dd-dab1c9232ff4 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:36.173040 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fbd372-120c-4426-afa2-ca0dfc6945ad admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:36.180345 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb52be4-85e3-43ac-9274-c347a37a5572 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd10b49b29df94404a71f43273839c252', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:36.181006 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb52be4-85e3-43ac-9274-c347a37a5572 service nova] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:36.181415 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb52be4-85e3-43ac-9274-c347a37a5572 service nova] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:36.181692 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb52be4-85e3-43ac-9274-c347a37a5572 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:36.200509 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb52be4-85e3-43ac-9274-c347a37a5572 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:36.203008 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb52be4-85e3-43ac-9274-c347a37a5572 service nova] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:36.218683 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 824/1678] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:44:35 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:44:36.309773 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-c1fbd372-120c-4426-afa2-ca0dfc6945ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f2ffc8d7e9044d2929416598eee79f6'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:44:36.326628 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-c1fbd372-120c-4426-afa2-ca0dfc6945ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f2ffc8d7e9044d2929416598eee79f6'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:44:36.327953 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-c1fbd372-120c-4426-afa2-ca0dfc6945ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f2ffc8d7e9044d2929416598eee79f6'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:44:36.335656 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:44:36.364697 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-c1fbd372-120c-4426-afa2-ca0dfc6945ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f2ffc8d7e9044d2929416598eee79f6'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:44:36.411452 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-c1fbd372-120c-4426-afa2-ca0dfc6945ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f2ffc8d7e9044d2929416598eee79f6'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:44:36.438864 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 855/1679] 10.222.0.89 () {58 vars in 1224 bytes} [Thu Jul 28 17:44:36 2022] DELETE /identity/v3/users/4f2ffc8d7e9044d2929416598eee79f6 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:44:37.002263 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb52be4-85e3-43ac-9274-c347a37a5572 service nova] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:37.025872 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb52be4-85e3-43ac-9274-c347a37a5572 service nova] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:37.053708 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:44:37.190212 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8544efa8-6e3d-4a24-90b0-186538acdaf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:37.190929 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8544efa8-6e3d-4a24-90b0-186538acdaf3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:37.196717 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8544efa8-6e3d-4a24-90b0-186538acdaf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:37.197058 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8544efa8-6e3d-4a24-90b0-186538acdaf3 admin admin] PATH_INFO: `/v3/projects/b708738fea994bae8807b3c0a93a0383` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:37.199015 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8544efa8-6e3d-4a24-90b0-186538acdaf3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b708738fea994bae8807b3c0a93a0383)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:37.204588 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8544efa8-6e3d-4a24-90b0-186538acdaf3 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:37.316277 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-8544efa8-6e3d-4a24-90b0-186538acdaf3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b708738fea994bae8807b3c0a93a0383', 'request_id': 'req-8544efa8-6e3d-4a24-90b0-186538acdaf3'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:44:37.328828 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-8544efa8-6e3d-4a24-90b0-186538acdaf3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b708738fea994bae8807b3c0a93a0383', 'request_id': 'req-8544efa8-6e3d-4a24-90b0-186538acdaf3'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:44:37.351848 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-8544efa8-6e3d-4a24-90b0-186538acdaf3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b708738fea994bae8807b3c0a93a0383', 'request_id': 'req-8544efa8-6e3d-4a24-90b0-186538acdaf3'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:44:37.380642 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-8544efa8-6e3d-4a24-90b0-186538acdaf3 admin admin] The token cache is being invalidate because project b708738fea994bae8807b3c0a93a0383 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80325) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 17:44:37.380642 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-8544efa8-6e3d-4a24-90b0-186538acdaf3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:44:37.381072 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-8544efa8-6e3d-4a24-90b0-186538acdaf3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:44:37.387605 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 825/1680] 10.222.0.89 () {58 vars in 1233 bytes} [Thu Jul 28 17:44:37 2022] DELETE /identity/v3/projects/b708738fea994bae8807b3c0a93a0383 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 17:44:37.388384 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fbd372-120c-4426-afa2-ca0dfc6945ad admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:37.391529 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-c1fbd372-120c-4426-afa2-ca0dfc6945ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:37.412869 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fbd372-120c-4426-afa2-ca0dfc6945ad admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:37.635308 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0bb445-9db3-4e0e-92d0-4ca1588b49c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:37.645744 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0bb445-9db3-4e0e-92d0-4ca1588b49c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:37.646454 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0bb445-9db3-4e0e-92d0-4ca1588b49c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:37.646710 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0bb445-9db3-4e0e-92d0-4ca1588b49c9 admin admin] PATH_INFO: `/v3/users/c4b0ff3bf2874b1abbc877f64475ba45` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:37.648374 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0bb445-9db3-4e0e-92d0-4ca1588b49c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c4b0ff3bf2874b1abbc877f64475ba45)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:37.650566 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0bb445-9db3-4e0e-92d0-4ca1588b49c9 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:37.795654 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-4c0bb445-9db3-4e0e-92d0-4ca1588b49c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c4b0ff3bf2874b1abbc877f64475ba45'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:44:37.838726 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-4c0bb445-9db3-4e0e-92d0-4ca1588b49c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c4b0ff3bf2874b1abbc877f64475ba45'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:44:37.845548 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-4c0bb445-9db3-4e0e-92d0-4ca1588b49c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c4b0ff3bf2874b1abbc877f64475ba45'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:44:37.875231 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-4c0bb445-9db3-4e0e-92d0-4ca1588b49c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c4b0ff3bf2874b1abbc877f64475ba45'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:44:37.897047 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-4c0bb445-9db3-4e0e-92d0-4ca1588b49c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c4b0ff3bf2874b1abbc877f64475ba45'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:44:37.914666 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 856/1681] 10.222.0.89 () {58 vars in 1224 bytes} [Thu Jul 28 17:44:37 2022] DELETE /identity/v3/users/c4b0ff3bf2874b1abbc877f64475ba45 => generated 0 bytes in 524 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:44:38.260095 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8544efa8-6e3d-4a24-90b0-186538acdaf3 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:38.261639 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-8544efa8-6e3d-4a24-90b0-186538acdaf3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:38.309307 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8544efa8-6e3d-4a24-90b0-186538acdaf3 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:38.460149 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0bb445-9db3-4e0e-92d0-4ca1588b49c9 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:38.461922 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-4c0bb445-9db3-4e0e-92d0-4ca1588b49c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:38.474522 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe61a9d-6874-4d58-9d2d-945d390cfaeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:38.477930 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe61a9d-6874-4d58-9d2d-945d390cfaeb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:38.478798 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe61a9d-6874-4d58-9d2d-945d390cfaeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:38.479245 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe61a9d-6874-4d58-9d2d-945d390cfaeb admin admin] PATH_INFO: `/v3/projects/414a3cb8d5af4b038fa928047d55e664` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:38.482029 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe61a9d-6874-4d58-9d2d-945d390cfaeb admin admin] RBAC: Authorizing `identity:delete_project(project_id=414a3cb8d5af4b038fa928047d55e664)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:38.484191 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe61a9d-6874-4d58-9d2d-945d390cfaeb admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:38.490236 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0bb445-9db3-4e0e-92d0-4ca1588b49c9 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:38.505283 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9b0d40-e4e9-4c4f-8e7e-ef56e0d7a883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:38.506507 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9b0d40-e4e9-4c4f-8e7e-ef56e0d7a883 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:38.506957 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9b0d40-e4e9-4c4f-8e7e-ef56e0d7a883 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:38.507422 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9b0d40-e4e9-4c4f-8e7e-ef56e0d7a883 admin admin] PATH_INFO: `/v3/users/ae0586e81d7e477ea9ae3efa55c8a4da` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:38.509646 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9b0d40-e4e9-4c4f-8e7e-ef56e0d7a883 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae0586e81d7e477ea9ae3efa55c8a4da)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:38.515562 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9b0d40-e4e9-4c4f-8e7e-ef56e0d7a883 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:38.623643 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:44:38.634112 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-7fe61a9d-6874-4d58-9d2d-945d390cfaeb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '414a3cb8d5af4b038fa928047d55e664', 'request_id': 'req-7fe61a9d-6874-4d58-9d2d-945d390cfaeb'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:44:38.643337 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-2d9b0d40-e4e9-4c4f-8e7e-ef56e0d7a883 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae0586e81d7e477ea9ae3efa55c8a4da'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:44:38.650905 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:44:38.662384 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-2d9b0d40-e4e9-4c4f-8e7e-ef56e0d7a883 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae0586e81d7e477ea9ae3efa55c8a4da'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:44:38.663195 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-2d9b0d40-e4e9-4c4f-8e7e-ef56e0d7a883 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae0586e81d7e477ea9ae3efa55c8a4da'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:44:38.665786 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-7fe61a9d-6874-4d58-9d2d-945d390cfaeb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '414a3cb8d5af4b038fa928047d55e664', 'request_id': 'req-7fe61a9d-6874-4d58-9d2d-945d390cfaeb'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:44:38.681732 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-2d9b0d40-e4e9-4c4f-8e7e-ef56e0d7a883 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae0586e81d7e477ea9ae3efa55c8a4da'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:44:38.688688 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-7fe61a9d-6874-4d58-9d2d-945d390cfaeb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '414a3cb8d5af4b038fa928047d55e664', 'request_id': 'req-7fe61a9d-6874-4d58-9d2d-945d390cfaeb'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:44:38.703605 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-2d9b0d40-e4e9-4c4f-8e7e-ef56e0d7a883 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae0586e81d7e477ea9ae3efa55c8a4da'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:44:38.710275 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-7fe61a9d-6874-4d58-9d2d-945d390cfaeb admin admin] The token cache is being invalidate because project 414a3cb8d5af4b038fa928047d55e664 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80325) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 17:44:38.710893 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-7fe61a9d-6874-4d58-9d2d-945d390cfaeb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:44:38.711941 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-7fe61a9d-6874-4d58-9d2d-945d390cfaeb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:44:38.716633 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 857/1683] 10.222.0.89 () {58 vars in 1224 bytes} [Thu Jul 28 17:44:38 2022] DELETE /identity/v3/users/ae0586e81d7e477ea9ae3efa55c8a4da => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:44:38.717488 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 826/1683] 10.222.0.89 () {58 vars in 1233 bytes} [Thu Jul 28 17:44:38 2022] DELETE /identity/v3/projects/414a3cb8d5af4b038fa928047d55e664 => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:44:38.718103 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7679004-85e0-403c-9b54-abc401c4cd4e None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:38.718820 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7679004-85e0-403c-9b54-abc401c4cd4e None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:38.718920 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe61a9d-6874-4d58-9d2d-945d390cfaeb admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:38.719009 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7679004-85e0-403c-9b54-abc401c4cd4e None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:38.719967 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 858/1684] 10.222.0.89 () {54 vars in 912 bytes} [Thu Jul 28 17:44:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:44:38.721062 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1042100-a3b6-49ae-ae65-5d6a7cd2a877 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:38.722346 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1042100-a3b6-49ae-ae65-5d6a7cd2a877 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:38.725242 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1042100-a3b6-49ae-ae65-5d6a7cd2a877 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:38.730892 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-7fe61a9d-6874-4d58-9d2d-945d390cfaeb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:38.779209 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-a1042100-a3b6-49ae-ae65-5d6a7cd2a877 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:44:38.781095 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-a1042100-a3b6-49ae-ae65-5d6a7cd2a877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:38.884955 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe61a9d-6874-4d58-9d2d-945d390cfaeb admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:38.944497 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 859/1685] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:44:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:44:38.945829 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b033069d-3be8-4299-b973-93157e6a9dd7 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:38.946114 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b033069d-3be8-4299-b973-93157e6a9dd7 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:38.946390 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b033069d-3be8-4299-b973-93157e6a9dd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:38.972614 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-b033069d-3be8-4299-b973-93157e6a9dd7 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:44:38.974035 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-b033069d-3be8-4299-b973-93157e6a9dd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:38.985739 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 860/1686] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:44:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:44:38.985739 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fe6540-c863-42f0-b877-524b7809e491 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:38.985739 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fe6540-c863-42f0-b877-524b7809e491 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:38.985739 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fe6540-c863-42f0-b877-524b7809e491 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:38.993841 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d892de2-ba12-44e9-ad49-8024af57ceb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:38.995068 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d892de2-ba12-44e9-ad49-8024af57ceb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:38.995456 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d892de2-ba12-44e9-ad49-8024af57ceb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:38.995838 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d892de2-ba12-44e9-ad49-8024af57ceb6 admin admin] PATH_INFO: `/v3/projects/c200bf1255164cc5aa6a81157a1eb894` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:38.997699 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d892de2-ba12-44e9-ad49-8024af57ceb6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c200bf1255164cc5aa6a81157a1eb894)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:39.001585 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d892de2-ba12-44e9-ad49-8024af57ceb6 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:39.054618 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-93fe6540-c863-42f0-b877-524b7809e491 None None] MFA Rules not processed for user `76817cd541294331b0023069092e8e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:44:39.055904 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-93fe6540-c863-42f0-b877-524b7809e491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:39.097410 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-7d892de2-ba12-44e9-ad49-8024af57ceb6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c200bf1255164cc5aa6a81157a1eb894', 'request_id': 'req-7d892de2-ba12-44e9-ad49-8024af57ceb6'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:44:39.106459 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-7d892de2-ba12-44e9-ad49-8024af57ceb6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c200bf1255164cc5aa6a81157a1eb894', 'request_id': 'req-7d892de2-ba12-44e9-ad49-8024af57ceb6'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:44:39.113478 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-7d892de2-ba12-44e9-ad49-8024af57ceb6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c200bf1255164cc5aa6a81157a1eb894', 'request_id': 'req-7d892de2-ba12-44e9-ad49-8024af57ceb6'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:44:39.139554 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-7d892de2-ba12-44e9-ad49-8024af57ceb6 admin admin] The token cache is being invalidate because project c200bf1255164cc5aa6a81157a1eb894 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80325) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 17:44:39.140715 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-7d892de2-ba12-44e9-ad49-8024af57ceb6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:44:39.143268 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-7d892de2-ba12-44e9-ad49-8024af57ceb6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:44:39.147706 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 827/1687] 10.222.0.89 () {58 vars in 1233 bytes} [Thu Jul 28 17:44:38 2022] DELETE /identity/v3/projects/c200bf1255164cc5aa6a81157a1eb894 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:44:39.149443 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef76c3b8-2b4e-4fb2-9906-8e8197ade06c None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:39.149916 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef76c3b8-2b4e-4fb2-9906-8e8197ade06c None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:39.150603 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef76c3b8-2b4e-4fb2-9906-8e8197ade06c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:39.182273 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 861/1688] 10.222.0.89 () {58 vars in 1009 bytes} [Thu Jul 28 17:44:38 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:44:39.184409 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-ef76c3b8-2b4e-4fb2-9906-8e8197ade06c None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:44:39.185144 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fe6540-c863-42f0-b877-524b7809e491 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:39.187118 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-ef76c3b8-2b4e-4fb2-9906-8e8197ade06c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:39.187755 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-93fe6540-c863-42f0-b877-524b7809e491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:39.203971 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:44:39.204416 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fe6540-c863-42f0-b877-524b7809e491 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:39.209702 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b43b81-b6e6-4981-992b-2a70b6cd691a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76817cd541294331b0023069092e8e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:39.212476 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b43b81-b6e6-4981-992b-2a70b6cd691a service neutron] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:39.212758 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b43b81-b6e6-4981-992b-2a70b6cd691a service neutron] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:39.213140 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b43b81-b6e6-4981-992b-2a70b6cd691a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:39.217208 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-b1b43b81-b6e6-4981-992b-2a70b6cd691a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:39.317355 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 828/1689] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:44:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:44:39.318536 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef76c3b8-2b4e-4fb2-9906-8e8197ade06c None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:39.320139 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-ef76c3b8-2b4e-4fb2-9906-8e8197ade06c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:39.338802 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef76c3b8-2b4e-4fb2-9906-8e8197ade06c None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:39.348858 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b63faf-5f16-46b8-8fe4-ca84db20901d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:39.350445 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b63faf-5f16-46b8-8fe4-ca84db20901d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:39.350811 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b63faf-5f16-46b8-8fe4-ca84db20901d admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:39.351322 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b63faf-5f16-46b8-8fe4-ca84db20901d admin admin] PATH_INFO: `/v3/users/005c03fdb4584f4ba7764d7246bfc5ea` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:39.353555 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b63faf-5f16-46b8-8fe4-ca84db20901d admin admin] RBAC: Authorizing `identity:delete_user(user_id=005c03fdb4584f4ba7764d7246bfc5ea)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:39.355831 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b63faf-5f16-46b8-8fe4-ca84db20901d admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:39.361008 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:44:39.367218 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b43b81-b6e6-4981-992b-2a70b6cd691a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:39.370335 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b43b81-b6e6-4981-992b-2a70b6cd691a service neutron] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:39.411285 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 862/1690] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:44:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:44:39.412648 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b43b81-b6e6-4981-992b-2a70b6cd691a service neutron] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:39.416479 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-b1b43b81-b6e6-4981-992b-2a70b6cd691a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:39.445123 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b43b81-b6e6-4981-992b-2a70b6cd691a service neutron] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:39.455641 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f263369e-43f2-4205-ab7b-bc7cca4625d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76817cd541294331b0023069092e8e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:39.457605 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f263369e-43f2-4205-ab7b-bc7cca4625d5 service neutron] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:39.457994 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f263369e-43f2-4205-ab7b-bc7cca4625d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:39.460344 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f263369e-43f2-4205-ab7b-bc7cca4625d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:39.479333 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f263369e-43f2-4205-ab7b-bc7cca4625d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:39.482924 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f263369e-43f2-4205-ab7b-bc7cca4625d5 service neutron] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:39.500806 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 863/1691] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:44:39 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:44:39.506145 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f263369e-43f2-4205-ab7b-bc7cca4625d5 service neutron] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:39.521176 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-d3b63faf-5f16-46b8-8fe4-ca84db20901d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '005c03fdb4584f4ba7764d7246bfc5ea'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:44:39.542384 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f263369e-43f2-4205-ab7b-bc7cca4625d5 service neutron] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:39.544148 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-d3b63faf-5f16-46b8-8fe4-ca84db20901d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '005c03fdb4584f4ba7764d7246bfc5ea'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:44:39.557448 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5c2694-483f-4c9a-90c3-64975245c934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:39.559914 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5c2694-483f-4c9a-90c3-64975245c934 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:39.560878 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5c2694-483f-4c9a-90c3-64975245c934 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:39.561263 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5c2694-483f-4c9a-90c3-64975245c934 admin admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:39.562816 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5c2694-483f-4c9a-90c3-64975245c934 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:44:39.565141 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5c2694-483f-4c9a-90c3-64975245c934 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:39.566041 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-d3b63faf-5f16-46b8-8fe4-ca84db20901d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '005c03fdb4584f4ba7764d7246bfc5ea'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:44:39.566783 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-d3b63faf-5f16-46b8-8fe4-ca84db20901d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '005c03fdb4584f4ba7764d7246bfc5ea'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:44:39.567878 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-d3b63faf-5f16-46b8-8fe4-ca84db20901d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '005c03fdb4584f4ba7764d7246bfc5ea'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:44:39.571030 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5c2694-483f-4c9a-90c3-64975245c934 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:39.614403 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 864/1692] 10.222.0.89 () {58 vars in 1153 bytes} [Thu Jul 28 17:44:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:44:39.619012 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5c2694-483f-4c9a-90c3-64975245c934 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:39.629020 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-db5c2694-483f-4c9a-90c3-64975245c934 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:39.642787 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 829/1693] 10.222.0.89 () {58 vars in 1224 bytes} [Thu Jul 28 17:44:39 2022] DELETE /identity/v3/users/005c03fdb4584f4ba7764d7246bfc5ea => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:44:39.669780 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5c2694-483f-4c9a-90c3-64975245c934 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:39.781520 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcd423f-11e5-46e6-a322-fc8a56df1882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:39.782495 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcd423f-11e5-46e6-a322-fc8a56df1882 admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:39.782792 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcd423f-11e5-46e6-a322-fc8a56df1882 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:39.783068 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcd423f-11e5-46e6-a322-fc8a56df1882 admin admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:39.783768 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcd423f-11e5-46e6-a322-fc8a56df1882 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:39.785769 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcd423f-11e5-46e6-a322-fc8a56df1882 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:39.834165 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 865/1694] 10.222.0.89 () {60 vars in 1153 bytes} [Thu Jul 28 17:44:39 2022] POST /identity/v3/projects => generated 371 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:44:39.838318 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b63faf-5f16-46b8-8fe4-ca84db20901d admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:39.853945 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b63faf-5f16-46b8-8fe4-ca84db20901d admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:39.997527 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a84f39d-f955-4302-b14b-599fe99a4d97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:39.998249 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a84f39d-f955-4302-b14b-599fe99a4d97 admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:39.998725 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a84f39d-f955-4302-b14b-599fe99a4d97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:39.999011 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a84f39d-f955-4302-b14b-599fe99a4d97 admin admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:40.001767 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a84f39d-f955-4302-b14b-599fe99a4d97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:40.003811 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a84f39d-f955-4302-b14b-599fe99a4d97 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:40.051654 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 830/1695] 10.222.0.89 () {60 vars in 1144 bytes} [Thu Jul 28 17:44:39 2022] POST /identity/v3/users => generated 286 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:44:40.052918 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcd423f-11e5-46e6-a322-fc8a56df1882 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:40.067737 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcd423f-11e5-46e6-a322-fc8a56df1882 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:40.081262 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c799d96f-64b2-4439-b38d-4c0ff0c74ef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:40.082200 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c799d96f-64b2-4439-b38d-4c0ff0c74ef6 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:40.082655 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c799d96f-64b2-4439-b38d-4c0ff0c74ef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:40.083099 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c799d96f-64b2-4439-b38d-4c0ff0c74ef6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:40.084008 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c799d96f-64b2-4439-b38d-4c0ff0c74ef6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:40.088388 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c799d96f-64b2-4439-b38d-4c0ff0c74ef6 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:40.116075 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 866/1696] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:44:40 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:44:40.117218 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a84f39d-f955-4302-b14b-599fe99a4d97 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:40.132491 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a84f39d-f955-4302-b14b-599fe99a4d97 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:40.146836 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63387f0-34a6-4749-8b6d-9aca495be42f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:40.147748 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63387f0-34a6-4749-8b6d-9aca495be42f admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:40.148251 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63387f0-34a6-4749-8b6d-9aca495be42f admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:40.148774 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63387f0-34a6-4749-8b6d-9aca495be42f admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:40.149713 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63387f0-34a6-4749-8b6d-9aca495be42f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:40.151528 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63387f0-34a6-4749-8b6d-9aca495be42f admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:40.170707 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:44:40.182135 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 831/1697] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:44:40 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 17:44:40.183079 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c799d96f-64b2-4439-b38d-4c0ff0c74ef6 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:40.194222 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c799d96f-64b2-4439-b38d-4c0ff0c74ef6 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:40.220465 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9604b8-c053-4416-8537-63439af9d4d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:40.221139 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9604b8-c053-4416-8537-63439af9d4d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:40.221475 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9604b8-c053-4416-8537-63439af9d4d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:40.221751 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9604b8-c053-4416-8537-63439af9d4d7 admin admin] PATH_INFO: `/v3/projects/e3bc2c9c7d254f65a104e3af382aa2d2/users/93fc422986f64994bf8eb865443c7c6c/roles/11f8731756fe4caca13efa61b71b11c5` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:40.253916 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9604b8-c053-4416-8537-63439af9d4d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3bc2c9c7d254f65a104e3af382aa2d2, user_id=93fc422986f64994bf8eb865443c7c6c, role_id=11f8731756fe4caca13efa61b71b11c5)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:40.256815 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9604b8-c053-4416-8537-63439af9d4d7 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:40.288349 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 867/1698] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:44:40 2022] PUT /identity/v3/projects/e3bc2c9c7d254f65a104e3af382aa2d2/users/93fc422986f64994bf8eb865443c7c6c/roles/11f8731756fe4caca13efa61b71b11c5 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 17:44:40.300047 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4fe9e9-627c-443b-b4f9-337d33b82d3b None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:40.304300 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4fe9e9-627c-443b-b4f9-337d33b82d3b None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:40.304848 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4fe9e9-627c-443b-b4f9-337d33b82d3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:40.364049 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-2f4fe9e9-627c-443b-b4f9-337d33b82d3b None None] MFA Rules not processed for user `93fc422986f64994bf8eb865443c7c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:44:40.365487 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-2f4fe9e9-627c-443b-b4f9-337d33b82d3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:40.542885 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 832/1699] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:44:40 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:44:40.547567 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9604b8-c053-4416-8537-63439af9d4d7 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:40.636858 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4fe9e9-627c-443b-b4f9-337d33b82d3b None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:40.642469 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-2f4fe9e9-627c-443b-b4f9-337d33b82d3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:40.720527 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9604b8-c053-4416-8537-63439af9d4d7 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:40.734385 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307010ee-ea54-4f0b-9e10-77c0ecf8ff5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:40.735458 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307010ee-ea54-4f0b-9e10-77c0ecf8ff5c admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:40.735738 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307010ee-ea54-4f0b-9e10-77c0ecf8ff5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:40.736012 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307010ee-ea54-4f0b-9e10-77c0ecf8ff5c admin admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:40.736699 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307010ee-ea54-4f0b-9e10-77c0ecf8ff5c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:40.743234 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307010ee-ea54-4f0b-9e10-77c0ecf8ff5c admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:40.815699 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4fe9e9-627c-443b-b4f9-337d33b82d3b None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:40.834908 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 868/1700] 10.222.0.89 () {60 vars in 1153 bytes} [Thu Jul 28 17:44:40 2022] POST /identity/v3/projects => generated 371 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:44:40.842459 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307010ee-ea54-4f0b-9e10-77c0ecf8ff5c admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:40.845340 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a1cd07-029a-4336-9f6b-2ded2f739176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:40.846132 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a1cd07-029a-4336-9f6b-2ded2f739176 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:40.846406 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a1cd07-029a-4336-9f6b-2ded2f739176 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:40.846687 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a1cd07-029a-4336-9f6b-2ded2f739176 admin admin] PATH_INFO: `/v3/users/3586e07a652b4f6f93c9448d9d1db885` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:40.858307 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a1cd07-029a-4336-9f6b-2ded2f739176 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3586e07a652b4f6f93c9448d9d1db885)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:40.860534 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a1cd07-029a-4336-9f6b-2ded2f739176 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:40.862906 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:44:41.028204 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-59a1cd07-029a-4336-9f6b-2ded2f739176 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3586e07a652b4f6f93c9448d9d1db885'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:44:41.049199 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-59a1cd07-029a-4336-9f6b-2ded2f739176 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3586e07a652b4f6f93c9448d9d1db885'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:44:41.056283 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307010ee-ea54-4f0b-9e10-77c0ecf8ff5c admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:41.063702 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a4f592-e899-4c7f-b39c-7f0b2405883c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:41.065148 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-59a1cd07-029a-4336-9f6b-2ded2f739176 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3586e07a652b4f6f93c9448d9d1db885'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:44:41.065913 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-59a1cd07-029a-4336-9f6b-2ded2f739176 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3586e07a652b4f6f93c9448d9d1db885'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:44:41.066583 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-59a1cd07-029a-4336-9f6b-2ded2f739176 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3586e07a652b4f6f93c9448d9d1db885'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:44:41.070025 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a4f592-e899-4c7f-b39c-7f0b2405883c admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:41.070025 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a4f592-e899-4c7f-b39c-7f0b2405883c admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:41.071509 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a4f592-e899-4c7f-b39c-7f0b2405883c admin admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:41.072341 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a4f592-e899-4c7f-b39c-7f0b2405883c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:41.075808 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a4f592-e899-4c7f-b39c-7f0b2405883c admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:41.103220 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 833/1701] 10.222.0.89 () {58 vars in 1224 bytes} [Thu Jul 28 17:44:40 2022] DELETE /identity/v3/users/3586e07a652b4f6f93c9448d9d1db885 => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:44:41.104464 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a1cd07-029a-4336-9f6b-2ded2f739176 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:41.106679 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-59a1cd07-029a-4336-9f6b-2ded2f739176 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:41.122628 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:44:41.127399 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 869/1702] 10.222.0.89 () {60 vars in 1144 bytes} [Thu Jul 28 17:44:40 2022] POST /identity/v3/users => generated 292 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:44:41.128390 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a4f592-e899-4c7f-b39c-7f0b2405883c admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:41.130266 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-d4a4f592-e899-4c7f-b39c-7f0b2405883c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:41.197424 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a1cd07-029a-4336-9f6b-2ded2f739176 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:41.203860 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776a9d76-7855-423c-a1f3-20bb1a0c0e51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:41.219318 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776a9d76-7855-423c-a1f3-20bb1a0c0e51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:41.219774 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776a9d76-7855-423c-a1f3-20bb1a0c0e51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:41.220267 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776a9d76-7855-423c-a1f3-20bb1a0c0e51 admin admin] PATH_INFO: `/v3/projects/1740f22f19344b8a91769cc0881dad7b` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:41.231024 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776a9d76-7855-423c-a1f3-20bb1a0c0e51 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1740f22f19344b8a91769cc0881dad7b)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:41.233107 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776a9d76-7855-423c-a1f3-20bb1a0c0e51 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:41.292092 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a4f592-e899-4c7f-b39c-7f0b2405883c admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:41.321427 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-776a9d76-7855-423c-a1f3-20bb1a0c0e51 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1740f22f19344b8a91769cc0881dad7b', 'request_id': 'req-776a9d76-7855-423c-a1f3-20bb1a0c0e51'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:44:41.336330 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-776a9d76-7855-423c-a1f3-20bb1a0c0e51 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1740f22f19344b8a91769cc0881dad7b', 'request_id': 'req-776a9d76-7855-423c-a1f3-20bb1a0c0e51'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:44:41.361429 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-776a9d76-7855-423c-a1f3-20bb1a0c0e51 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1740f22f19344b8a91769cc0881dad7b', 'request_id': 'req-776a9d76-7855-423c-a1f3-20bb1a0c0e51'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:44:41.387599 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-776a9d76-7855-423c-a1f3-20bb1a0c0e51 admin admin] The token cache is being invalidate because project 1740f22f19344b8a91769cc0881dad7b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80325) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 17:44:41.388245 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-776a9d76-7855-423c-a1f3-20bb1a0c0e51 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:44:41.393614 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-776a9d76-7855-423c-a1f3-20bb1a0c0e51 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:44:41.395728 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 834/1703] 10.222.0.89 () {58 vars in 1233 bytes} [Thu Jul 28 17:44:41 2022] DELETE /identity/v3/projects/1740f22f19344b8a91769cc0881dad7b => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:44:41.397111 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040471ca-39ac-4a41-92a5-ac43b117e504 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:41.397422 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040471ca-39ac-4a41-92a5-ac43b117e504 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:41.397695 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040471ca-39ac-4a41-92a5-ac43b117e504 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:41.406489 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de0d659-bab9-4360-b7a3-049a440e880c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4bd290018394b0d955cd376ce35b77f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:41.408682 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de0d659-bab9-4360-b7a3-049a440e880c service placement] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:41.408963 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de0d659-bab9-4360-b7a3-049a440e880c service placement] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:41.415060 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de0d659-bab9-4360-b7a3-049a440e880c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:41.431323 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-0de0d659-bab9-4360-b7a3-049a440e880c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:41.447275 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de0d659-bab9-4360-b7a3-049a440e880c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:41.449042 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de0d659-bab9-4360-b7a3-049a440e880c service placement] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:41.462543 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 870/1704] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:44:41 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:44:41.472660 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-040471ca-39ac-4a41-92a5-ac43b117e504 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:44:41.474997 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-040471ca-39ac-4a41-92a5-ac43b117e504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:41.483219 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de0d659-bab9-4360-b7a3-049a440e880c service placement] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:41.487608 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-0de0d659-bab9-4360-b7a3-049a440e880c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:41.582183 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:44:41.623774 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 835/1705] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:44:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:44:41.628423 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bfefc3-a8fe-4707-aab3-e9c7e701a7a9 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:41.631008 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bfefc3-a8fe-4707-aab3-e9c7e701a7a9 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:41.632580 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bfefc3-a8fe-4707-aab3-e9c7e701a7a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:41.684004 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-73bfefc3-a8fe-4707-aab3-e9c7e701a7a9 None None] MFA Rules not processed for user `14327b8153ab4de782d4bcfef7681fc1`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:44:41.690143 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-73bfefc3-a8fe-4707-aab3-e9c7e701a7a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:41.723553 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de0d659-bab9-4360-b7a3-049a440e880c service placement] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:41.744164 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f000b53c-51ab-4e7b-b304-36630b2589ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:41.747719 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f000b53c-51ab-4e7b-b304-36630b2589ab admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:41.748303 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f000b53c-51ab-4e7b-b304-36630b2589ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:41.749696 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f000b53c-51ab-4e7b-b304-36630b2589ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:41.763822 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f000b53c-51ab-4e7b-b304-36630b2589ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:41.769213 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f000b53c-51ab-4e7b-b304-36630b2589ab admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:41.786974 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 871/1706] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:44:41 2022] GET /identity/v3/roles => generated 1420 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:44:41.788645 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1adf4ab-ddda-425e-8e37-0f7646652a74 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:41.788969 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1adf4ab-ddda-425e-8e37-0f7646652a74 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:41.789372 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1adf4ab-ddda-425e-8e37-0f7646652a74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:41.841103 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:44:41.859601 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-a1adf4ab-ddda-425e-8e37-0f7646652a74 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:44:41.861087 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-a1adf4ab-ddda-425e-8e37-0f7646652a74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:41.893304 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 872/1707] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:44:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:44:41.900221 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1adf4ab-ddda-425e-8e37-0f7646652a74 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:41.901755 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-a1adf4ab-ddda-425e-8e37-0f7646652a74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:41.902211 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 836/1708] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:44:41 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:44:41.903980 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0cd787-e866-4a0e-b3bf-cd5157dacf03 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:41.903980 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0cd787-e866-4a0e-b3bf-cd5157dacf03 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:41.904173 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0cd787-e866-4a0e-b3bf-cd5157dacf03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:41.932052 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1adf4ab-ddda-425e-8e37-0f7646652a74 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:41.954063 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3f1039-827e-4f72-8518-77e75f9c06cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:41.954929 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3f1039-827e-4f72-8518-77e75f9c06cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:41.955358 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3f1039-827e-4f72-8518-77e75f9c06cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:41.956472 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3f1039-827e-4f72-8518-77e75f9c06cd admin admin] PATH_INFO: `/v3/projects/db52bab56abd4dd98539858e1c5eefa8` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:41.961378 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3f1039-827e-4f72-8518-77e75f9c06cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=db52bab56abd4dd98539858e1c5eefa8)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:41.964627 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: WARNING py.warnings [None req-5e3f1039-827e-4f72-8518-77e75f9c06cd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 28 17:44:41.964627 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: warnings.warn(msg) Jul 28 17:44:41.964627 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: Jul 28 17:44:41.970601 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3f1039-827e-4f72-8518-77e75f9c06cd admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:41.975181 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-4e0cd787-e866-4a0e-b3bf-cd5157dacf03 None None] MFA Rules not processed for user `22479a66b0f34038b144f4def1aa3f43`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:44:41.977327 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-4e0cd787-e866-4a0e-b3bf-cd5157dacf03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:42.150590 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 837/1709] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:44:41 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:44:42.151806 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0cd787-e866-4a0e-b3bf-cd5157dacf03 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:42.160320 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-4e0cd787-e866-4a0e-b3bf-cd5157dacf03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:42.276425 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-5e3f1039-827e-4f72-8518-77e75f9c06cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db52bab56abd4dd98539858e1c5eefa8', 'request_id': 'req-5e3f1039-827e-4f72-8518-77e75f9c06cd'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:44:42.323701 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-5e3f1039-827e-4f72-8518-77e75f9c06cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db52bab56abd4dd98539858e1c5eefa8', 'request_id': 'req-5e3f1039-827e-4f72-8518-77e75f9c06cd'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:44:42.333949 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0cd787-e866-4a0e-b3bf-cd5157dacf03 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:42.387993 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-5e3f1039-827e-4f72-8518-77e75f9c06cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db52bab56abd4dd98539858e1c5eefa8', 'request_id': 'req-5e3f1039-827e-4f72-8518-77e75f9c06cd'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:44:42.417393 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-5e3f1039-827e-4f72-8518-77e75f9c06cd admin admin] The token cache is being invalidate because project db52bab56abd4dd98539858e1c5eefa8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80324) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 17:44:42.417845 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-5e3f1039-827e-4f72-8518-77e75f9c06cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:44:42.418830 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-5e3f1039-827e-4f72-8518-77e75f9c06cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:44:42.427041 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 873/1710] 10.222.0.89 () {58 vars in 1233 bytes} [Thu Jul 28 17:44:41 2022] DELETE /identity/v3/projects/db52bab56abd4dd98539858e1c5eefa8 => generated 0 bytes in 529 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:44:42.430221 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3f1039-827e-4f72-8518-77e75f9c06cd admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:42.438358 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-5e3f1039-827e-4f72-8518-77e75f9c06cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:42.464337 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c09e25-4c1e-4e55-911c-1627ef08b68b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76817cd541294331b0023069092e8e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:42.465121 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c09e25-4c1e-4e55-911c-1627ef08b68b service neutron] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:42.470226 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c09e25-4c1e-4e55-911c-1627ef08b68b service neutron] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:42.470668 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c09e25-4c1e-4e55-911c-1627ef08b68b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:42.473846 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-15c09e25-4c1e-4e55-911c-1627ef08b68b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:42.604370 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3f1039-827e-4f72-8518-77e75f9c06cd admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:42.606313 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:44:42.621753 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c731c5ab-ba4d-4441-8b3d-845b7061ebcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:42.623028 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c731c5ab-ba4d-4441-8b3d-845b7061ebcc admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:42.623822 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c731c5ab-ba4d-4441-8b3d-845b7061ebcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:42.624274 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c731c5ab-ba4d-4441-8b3d-845b7061ebcc admin admin] PATH_INFO: `/v3/projects/26c7ea56f91b405ebd81f2aae73242b7/users/83c6c3493f954c94abe2e7c3223d5974/roles/73e457231a824ea98c28d22610be640c` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:42.662188 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c09e25-4c1e-4e55-911c-1627ef08b68b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:42.663029 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c731c5ab-ba4d-4441-8b3d-845b7061ebcc admin admin] RBAC: Authorizing `identity:create_grant(project_id=26c7ea56f91b405ebd81f2aae73242b7, user_id=83c6c3493f954c94abe2e7c3223d5974, role_id=73e457231a824ea98c28d22610be640c)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:42.665864 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c731c5ab-ba4d-4441-8b3d-845b7061ebcc admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:42.666829 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c09e25-4c1e-4e55-911c-1627ef08b68b service neutron] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:42.691496 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 874/1711] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:44:42 2022] PUT /identity/v3/projects/26c7ea56f91b405ebd81f2aae73242b7/users/83c6c3493f954c94abe2e7c3223d5974/roles/73e457231a824ea98c28d22610be640c => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:44:42.693306 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b58b6e-d724-4ad5-ab36-8bd87b5b8597 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:42.695020 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 838/1712] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:44:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 544 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:44:42.695356 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b58b6e-d724-4ad5-ab36-8bd87b5b8597 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:42.695746 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b58b6e-d724-4ad5-ab36-8bd87b5b8597 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:42.695939 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c09e25-4c1e-4e55-911c-1627ef08b68b service neutron] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:42.697972 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-15c09e25-4c1e-4e55-911c-1627ef08b68b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:42.742478 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-d5b58b6e-d724-4ad5-ab36-8bd87b5b8597 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:44:42.745075 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-d5b58b6e-d724-4ad5-ab36-8bd87b5b8597 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:42.782722 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:44:42.891848 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 875/1713] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:44:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:44:42.896300 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6950ecb-e299-4041-9964-a3027cc4b814 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:42.896848 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c09e25-4c1e-4e55-911c-1627ef08b68b service neutron] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:42.898203 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6950ecb-e299-4041-9964-a3027cc4b814 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:42.898697 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6950ecb-e299-4041-9964-a3027cc4b814 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:42.944871 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-f6950ecb-e299-4041-9964-a3027cc4b814 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:44:42.946271 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-f6950ecb-e299-4041-9964-a3027cc4b814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:42.975090 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 876/1714] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:44:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:44:42.989514 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6950ecb-e299-4041-9964-a3027cc4b814 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:43.022005 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6950ecb-e299-4041-9964-a3027cc4b814 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:43.041330 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d47838b-11c9-424f-8be3-863a5512ca57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:43.043536 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d47838b-11c9-424f-8be3-863a5512ca57 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:43.047586 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d47838b-11c9-424f-8be3-863a5512ca57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:43.048017 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d47838b-11c9-424f-8be3-863a5512ca57 admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:43.048880 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d47838b-11c9-424f-8be3-863a5512ca57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:43.051486 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d47838b-11c9-424f-8be3-863a5512ca57 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:43.071251 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 877/1715] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:44:42 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:44:43.097953 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d47838b-11c9-424f-8be3-863a5512ca57 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:43.134477 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d47838b-11c9-424f-8be3-863a5512ca57 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:43.148954 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da54fb97-30c2-477e-ab84-27c0327bc0bd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd10b49b29df94404a71f43273839c252', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:43.150466 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da54fb97-30c2-477e-ab84-27c0327bc0bd service nova] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:43.151968 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da54fb97-30c2-477e-ab84-27c0327bc0bd service nova] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:43.152471 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da54fb97-30c2-477e-ab84-27c0327bc0bd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:43.153322 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68e5e17-112c-409a-b5c8-16801f4754fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:43.156010 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68e5e17-112c-409a-b5c8-16801f4754fc admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:43.163722 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68e5e17-112c-409a-b5c8-16801f4754fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:43.165023 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-da54fb97-30c2-477e-ab84-27c0327bc0bd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:43.165988 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68e5e17-112c-409a-b5c8-16801f4754fc admin admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:43.167520 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68e5e17-112c-409a-b5c8-16801f4754fc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:44:43.168156 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68e5e17-112c-409a-b5c8-16801f4754fc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:43.170451 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68e5e17-112c-409a-b5c8-16801f4754fc admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:43.218654 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 878/1716] 10.222.0.89 () {58 vars in 1153 bytes} [Thu Jul 28 17:44:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:44:43.225171 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbde6c3-01f6-4e38-a119-c3aef5012bca None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:43.227076 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbde6c3-01f6-4e38-a119-c3aef5012bca None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:43.227457 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbde6c3-01f6-4e38-a119-c3aef5012bca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:43.300419 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-bbbde6c3-01f6-4e38-a119-c3aef5012bca None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:44:43.305887 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-bbbde6c3-01f6-4e38-a119-c3aef5012bca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:43.341299 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 879/1717] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:44:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:44:43.343107 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbde6c3-01f6-4e38-a119-c3aef5012bca None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:43.383948 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbde6c3-01f6-4e38-a119-c3aef5012bca None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:43.394157 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da54fb97-30c2-477e-ab84-27c0327bc0bd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:43.396263 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da54fb97-30c2-477e-ab84-27c0327bc0bd service nova] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:43.404553 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd6e1d5-5a8b-4a02-ad99-1191a89ea9ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:43.406163 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd6e1d5-5a8b-4a02-ad99-1191a89ea9ca admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:43.406714 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd6e1d5-5a8b-4a02-ad99-1191a89ea9ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:43.407211 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd6e1d5-5a8b-4a02-ad99-1191a89ea9ca admin admin] PATH_INFO: `/v3/projects/26c7ea56f91b405ebd81f2aae73242b7/users/83c6c3493f954c94abe2e7c3223d5974/roles/73e457231a824ea98c28d22610be640c` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:43.410499 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd6e1d5-5a8b-4a02-ad99-1191a89ea9ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=26c7ea56f91b405ebd81f2aae73242b7, user_id=83c6c3493f954c94abe2e7c3223d5974, role_id=73e457231a824ea98c28d22610be640c)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:43.415180 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd6e1d5-5a8b-4a02-ad99-1191a89ea9ca admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:43.419227 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 839/1718] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:44:42 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 724 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:44:43.421901 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a546d08-83ba-43e4-a7bb-d7e54aac2457 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:43.423438 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a546d08-83ba-43e4-a7bb-d7e54aac2457 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:43.423860 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a546d08-83ba-43e4-a7bb-d7e54aac2457 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:43.442884 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 880/1719] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:44:43 2022] PUT /identity/v3/projects/26c7ea56f91b405ebd81f2aae73242b7/users/83c6c3493f954c94abe2e7c3223d5974/roles/73e457231a824ea98c28d22610be640c => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:44:43.444832 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd6e1d5-5a8b-4a02-ad99-1191a89ea9ca admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:43.500848 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-2a546d08-83ba-43e4-a7bb-d7e54aac2457 None None] MFA Rules not processed for user `0b97c60c0360466fb878e1e9141e1f3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:44:43.502490 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-2a546d08-83ba-43e4-a7bb-d7e54aac2457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:43.514071 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:44:43.601776 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd6e1d5-5a8b-4a02-ad99-1191a89ea9ca admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:43.618608 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cb26fc-6dd8-4324-af38-ce3ad4fdaf1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:43.620484 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cb26fc-6dd8-4324-af38-ce3ad4fdaf1c admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:43.621598 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cb26fc-6dd8-4324-af38-ce3ad4fdaf1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:43.622145 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cb26fc-6dd8-4324-af38-ce3ad4fdaf1c admin admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:43.624044 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cb26fc-6dd8-4324-af38-ce3ad4fdaf1c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:43.630229 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cb26fc-6dd8-4324-af38-ce3ad4fdaf1c admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:43.692138 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 840/1720] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:44:43 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:44:43.701929 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a546d08-83ba-43e4-a7bb-d7e54aac2457 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:43.739261 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 881/1721] 10.222.0.89 () {60 vars in 1153 bytes} [Thu Jul 28 17:44:43 2022] POST /identity/v3/projects => generated 371 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:44:43.739261 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4416ba-4fb2-4764-becd-27f85edf6d99 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:43.741256 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4416ba-4fb2-4764-becd-27f85edf6d99 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:43.743077 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4416ba-4fb2-4764-becd-27f85edf6d99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:43.751757 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a546d08-83ba-43e4-a7bb-d7e54aac2457 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:43.862586 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-9b4416ba-4fb2-4764-becd-27f85edf6d99 None None] MFA Rules not processed for user `83c6c3493f954c94abe2e7c3223d5974`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:44:43.863974 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-9b4416ba-4fb2-4764-becd-27f85edf6d99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:43.965963 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f6527f-25b8-4c33-8fc3-54debeb3c2b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:43.966917 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f6527f-25b8-4c33-8fc3-54debeb3c2b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:43.967406 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f6527f-25b8-4c33-8fc3-54debeb3c2b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:43.967782 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f6527f-25b8-4c33-8fc3-54debeb3c2b6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:43.969273 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f6527f-25b8-4c33-8fc3-54debeb3c2b6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:44:43.969704 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f6527f-25b8-4c33-8fc3-54debeb3c2b6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:43.975022 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f6527f-25b8-4c33-8fc3-54debeb3c2b6 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:44.008220 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 841/1722] 10.222.0.89 () {58 vars in 1153 bytes} [Thu Jul 28 17:44:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:44:44.009882 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f6527f-25b8-4c33-8fc3-54debeb3c2b6 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:44.032136 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f6527f-25b8-4c33-8fc3-54debeb3c2b6 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:44.046716 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:44:44.051246 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfa219f-a2b9-496f-9378-eb0c07d57e80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:44.052016 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfa219f-a2b9-496f-9378-eb0c07d57e80 admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:44.052412 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfa219f-a2b9-496f-9378-eb0c07d57e80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:44.052786 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfa219f-a2b9-496f-9378-eb0c07d57e80 admin admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:44.053758 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfa219f-a2b9-496f-9378-eb0c07d57e80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:44.066970 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfa219f-a2b9-496f-9378-eb0c07d57e80 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:44.136872 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 882/1723] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:44:43 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 400 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:44:44.137865 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4416ba-4fb2-4764-becd-27f85edf6d99 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:44.146586 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 842/1724] 10.222.0.89 () {60 vars in 1144 bytes} [Thu Jul 28 17:44:44 2022] POST /identity/v3/users => generated 286 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:44:44.147851 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfa219f-a2b9-496f-9378-eb0c07d57e80 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:44.173094 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfa219f-a2b9-496f-9378-eb0c07d57e80 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:44.189783 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890b73a8-e68e-4b80-bc97-44915b68fd80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:44.190732 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890b73a8-e68e-4b80-bc97-44915b68fd80 admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:44.191202 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890b73a8-e68e-4b80-bc97-44915b68fd80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:44.191741 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890b73a8-e68e-4b80-bc97-44915b68fd80 admin admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:44.194714 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890b73a8-e68e-4b80-bc97-44915b68fd80 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:44.200292 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890b73a8-e68e-4b80-bc97-44915b68fd80 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:44.313365 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 843/1725] 10.222.0.89 () {60 vars in 1153 bytes} [Thu Jul 28 17:44:44 2022] POST /identity/v3/projects => generated 383 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:44:44.317332 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890b73a8-e68e-4b80-bc97-44915b68fd80 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:44.323519 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-890b73a8-e68e-4b80-bc97-44915b68fd80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:44.423175 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4416ba-4fb2-4764-becd-27f85edf6d99 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:44.555264 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890b73a8-e68e-4b80-bc97-44915b68fd80 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:44.571288 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8041209-8e30-44c1-9366-3e4411f17f28 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96d512d0c84240918dae6c4a3e8b7b79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:44.586824 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8041209-8e30-44c1-9366-3e4411f17f28 service cinder] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:44.589275 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8041209-8e30-44c1-9366-3e4411f17f28 service cinder] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:44.590909 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8041209-8e30-44c1-9366-3e4411f17f28 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:44.675927 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:44:44.725943 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b933176-4cc5-420a-9e26-d4915f746377 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd10b49b29df94404a71f43273839c252', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:44.726889 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b933176-4cc5-420a-9e26-d4915f746377 service nova] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:44.727178 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b933176-4cc5-420a-9e26-d4915f746377 service nova] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:44.727458 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b933176-4cc5-420a-9e26-d4915f746377 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:44.857781 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8041209-8e30-44c1-9366-3e4411f17f28 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:44.859338 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8041209-8e30-44c1-9366-3e4411f17f28 service cinder] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:44.893408 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 844/1726] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:44:44 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 579 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:44:44.916705 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8041209-8e30-44c1-9366-3e4411f17f28 service cinder] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:44.957828 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b933176-4cc5-420a-9e26-d4915f746377 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:44.959757 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b933176-4cc5-420a-9e26-d4915f746377 service nova] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:45.004045 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 883/1727] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:44:44 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 867 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:44:45.020889 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b1ada8-09a3-4dae-83c1-d16cd77c71e2 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:45.021618 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b1ada8-09a3-4dae-83c1-d16cd77c71e2 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:45.021618 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b1ada8-09a3-4dae-83c1-d16cd77c71e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:45.101300 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8041209-8e30-44c1-9366-3e4411f17f28 service cinder] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:45.118731 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e8fb31-4ecc-42f6-8be9-d2224dfd7a48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:45.119464 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e8fb31-4ecc-42f6-8be9-d2224dfd7a48 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:45.119636 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e8fb31-4ecc-42f6-8be9-d2224dfd7a48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:45.122220 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e8fb31-4ecc-42f6-8be9-d2224dfd7a48 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:45.122974 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e8fb31-4ecc-42f6-8be9-d2224dfd7a48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:45.131757 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e8fb31-4ecc-42f6-8be9-d2224dfd7a48 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:45.172134 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-24b1ada8-09a3-4dae-83c1-d16cd77c71e2 None None] MFA Rules not processed for user `6ed334d56f34486ebd8b52247d060856`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:44:45.178337 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-24b1ada8-09a3-4dae-83c1-d16cd77c71e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:45.185908 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 845/1728] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:44:44 2022] GET /identity/v3/roles => generated 1420 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:44:45.197598 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e8fb31-4ecc-42f6-8be9-d2224dfd7a48 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:45.200204 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:44:45.262627 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e8fb31-4ecc-42f6-8be9-d2224dfd7a48 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:45.295420 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baacacc0-2d8a-427b-ac76-1452f06d599e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:45.306214 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baacacc0-2d8a-427b-ac76-1452f06d599e admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:45.307459 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baacacc0-2d8a-427b-ac76-1452f06d599e admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:45.307896 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baacacc0-2d8a-427b-ac76-1452f06d599e admin admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:45.309666 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baacacc0-2d8a-427b-ac76-1452f06d599e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:45.313818 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baacacc0-2d8a-427b-ac76-1452f06d599e admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:45.404680 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 846/1729] 10.222.0.89 () {60 vars in 1144 bytes} [Thu Jul 28 17:44:45 2022] POST /identity/v3/users => generated 292 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:44:45.405587 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baacacc0-2d8a-427b-ac76-1452f06d599e admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:45.413354 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-baacacc0-2d8a-427b-ac76-1452f06d599e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:45.457202 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 884/1730] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:44:45 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 437 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:44:45.458495 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b1ada8-09a3-4dae-83c1-d16cd77c71e2 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:45.473232 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baacacc0-2d8a-427b-ac76-1452f06d599e admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:45.488448 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc416f2-2605-4f4e-bfbe-291e38cb2d0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:45.490775 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc416f2-2605-4f4e-bfbe-291e38cb2d0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:45.491242 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc416f2-2605-4f4e-bfbe-291e38cb2d0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:45.492650 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc416f2-2605-4f4e-bfbe-291e38cb2d0a admin admin] PATH_INFO: `/v3/projects/bdc42a1f064b4cf0acb0d9ce3dee6b94` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:45.496113 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc416f2-2605-4f4e-bfbe-291e38cb2d0a admin admin] RBAC: Authorizing `identity:delete_project(project_id=bdc42a1f064b4cf0acb0d9ce3dee6b94)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:45.498949 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc416f2-2605-4f4e-bfbe-291e38cb2d0a admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:45.506335 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b1ada8-09a3-4dae-83c1-d16cd77c71e2 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:45.528563 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e78897-4bb6-424d-b0be-0497c9dccdd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:45.535350 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e78897-4bb6-424d-b0be-0497c9dccdd5 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:45.536470 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e78897-4bb6-424d-b0be-0497c9dccdd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:45.552777 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e78897-4bb6-424d-b0be-0497c9dccdd5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:45.553504 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e78897-4bb6-424d-b0be-0497c9dccdd5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:45.555397 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e78897-4bb6-424d-b0be-0497c9dccdd5 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:45.585267 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 885/1731] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:44:45 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:44:45.589100 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca673971-7d45-47e1-8b96-292c9f70f1d1 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:45.589467 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca673971-7d45-47e1-8b96-292c9f70f1d1 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:45.589749 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca673971-7d45-47e1-8b96-292c9f70f1d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:45.601275 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-4dc416f2-2605-4f4e-bfbe-291e38cb2d0a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bdc42a1f064b4cf0acb0d9ce3dee6b94', 'request_id': 'req-4dc416f2-2605-4f4e-bfbe-291e38cb2d0a'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:44:45.620690 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-4dc416f2-2605-4f4e-bfbe-291e38cb2d0a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bdc42a1f064b4cf0acb0d9ce3dee6b94', 'request_id': 'req-4dc416f2-2605-4f4e-bfbe-291e38cb2d0a'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:44:45.642410 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-4dc416f2-2605-4f4e-bfbe-291e38cb2d0a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bdc42a1f064b4cf0acb0d9ce3dee6b94', 'request_id': 'req-4dc416f2-2605-4f4e-bfbe-291e38cb2d0a'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:44:45.690572 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-4dc416f2-2605-4f4e-bfbe-291e38cb2d0a admin admin] The token cache is being invalidate because project bdc42a1f064b4cf0acb0d9ce3dee6b94 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80325) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 17:44:45.690878 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-4dc416f2-2605-4f4e-bfbe-291e38cb2d0a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:44:45.691644 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-4dc416f2-2605-4f4e-bfbe-291e38cb2d0a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:44:45.693693 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 847/1732] 10.222.0.89 () {58 vars in 1233 bytes} [Thu Jul 28 17:44:45 2022] DELETE /identity/v3/projects/bdc42a1f064b4cf0acb0d9ce3dee6b94 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:44:45.694870 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc416f2-2605-4f4e-bfbe-291e38cb2d0a admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:45.696808 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-4dc416f2-2605-4f4e-bfbe-291e38cb2d0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:45.732211 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:44:45.759558 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-ca673971-7d45-47e1-8b96-292c9f70f1d1 None None] MFA Rules not processed for user `c44387d13a694ee6bb47a13906003330`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:44:45.764429 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-ca673971-7d45-47e1-8b96-292c9f70f1d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:45.850347 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc416f2-2605-4f4e-bfbe-291e38cb2d0a admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:45.855591 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce705dac-f223-429e-a3e0-deb5f47be98b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:45.856651 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce705dac-f223-429e-a3e0-deb5f47be98b admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:45.856945 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce705dac-f223-429e-a3e0-deb5f47be98b admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:45.857254 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce705dac-f223-429e-a3e0-deb5f47be98b admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:45.858105 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce705dac-f223-429e-a3e0-deb5f47be98b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:45.859882 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce705dac-f223-429e-a3e0-deb5f47be98b admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:45.888527 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 848/1733] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:44:45 2022] GET /identity/v3/roles => generated 1420 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:44:45.891977 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cf6c3c-3fbc-453f-a2bc-af7d24e49937 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:45.892337 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cf6c3c-3fbc-453f-a2bc-af7d24e49937 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:45.892667 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cf6c3c-3fbc-453f-a2bc-af7d24e49937 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:45.915625 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-93cf6c3c-3fbc-453f-a2bc-af7d24e49937 None None] MFA Rules not processed for user `6ed334d56f34486ebd8b52247d060856`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:44:45.916963 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-93cf6c3c-3fbc-453f-a2bc-af7d24e49937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:46.009896 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 849/1734] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:44:45 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:44:46.010608 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cf6c3c-3fbc-453f-a2bc-af7d24e49937 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:46.012080 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-93cf6c3c-3fbc-453f-a2bc-af7d24e49937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:46.033091 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cf6c3c-3fbc-453f-a2bc-af7d24e49937 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:46.040411 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb569d97-e573-40f8-a514-eac1720035d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:46.041584 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb569d97-e573-40f8-a514-eac1720035d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:46.041865 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb569d97-e573-40f8-a514-eac1720035d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:46.042585 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb569d97-e573-40f8-a514-eac1720035d6 admin admin] PATH_INFO: `/v3/projects/67c07aa592e24366b3c9d844ea20085b/users/6bfb0add8a7248e692f177ae58d782e3/roles/11f8731756fe4caca13efa61b71b11c5` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:46.071724 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb569d97-e573-40f8-a514-eac1720035d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=67c07aa592e24366b3c9d844ea20085b, user_id=6bfb0add8a7248e692f177ae58d782e3, role_id=11f8731756fe4caca13efa61b71b11c5)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:46.075737 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb569d97-e573-40f8-a514-eac1720035d6 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:46.095573 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 850/1735] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:44:46 2022] PUT /identity/v3/projects/67c07aa592e24366b3c9d844ea20085b/users/6bfb0add8a7248e692f177ae58d782e3/roles/11f8731756fe4caca13efa61b71b11c5 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:44:46.096224 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 886/1736] 10.222.0.89 () {58 vars in 1009 bytes} [Thu Jul 28 17:44:45 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 511 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:44:46.096883 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb569d97-e573-40f8-a514-eac1720035d6 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:46.097191 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca673971-7d45-47e1-8b96-292c9f70f1d1 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:46.108203 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-fb569d97-e573-40f8-a514-eac1720035d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:46.139152 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca673971-7d45-47e1-8b96-292c9f70f1d1 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:46.354559 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb569d97-e573-40f8-a514-eac1720035d6 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:46.374055 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a55e89-49f2-4e0a-9b43-41f804de8e79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:46.378934 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46de2213-8553-4f28-af93-12bdfba8017e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:46.379602 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46de2213-8553-4f28-af93-12bdfba8017e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:46.379882 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46de2213-8553-4f28-af93-12bdfba8017e admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:46.380155 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46de2213-8553-4f28-af93-12bdfba8017e admin admin] PATH_INFO: `/v3/users/9448c75844b24f5fb671d96d6164fad9` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:46.382435 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a55e89-49f2-4e0a-9b43-41f804de8e79 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:46.382781 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a55e89-49f2-4e0a-9b43-41f804de8e79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:46.387120 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a55e89-49f2-4e0a-9b43-41f804de8e79 admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:46.387901 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a55e89-49f2-4e0a-9b43-41f804de8e79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:46.392803 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46de2213-8553-4f28-af93-12bdfba8017e admin admin] RBAC: Authorizing `identity:delete_user(user_id=9448c75844b24f5fb671d96d6164fad9)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:46.397536 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a55e89-49f2-4e0a-9b43-41f804de8e79 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:46.402192 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46de2213-8553-4f28-af93-12bdfba8017e admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:46.464260 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 887/1737] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:44:46 2022] GET /identity/v3/roles => generated 1420 bytes in 365 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:44:46.466052 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a55e89-49f2-4e0a-9b43-41f804de8e79 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:46.474926 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-74a55e89-49f2-4e0a-9b43-41f804de8e79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:46.497653 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:44:46.549432 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:44:46.581736 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-46de2213-8553-4f28-af93-12bdfba8017e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9448c75844b24f5fb671d96d6164fad9'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:44:46.598985 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-46de2213-8553-4f28-af93-12bdfba8017e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9448c75844b24f5fb671d96d6164fad9'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:44:46.628034 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-46de2213-8553-4f28-af93-12bdfba8017e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9448c75844b24f5fb671d96d6164fad9'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:44:46.629300 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-46de2213-8553-4f28-af93-12bdfba8017e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9448c75844b24f5fb671d96d6164fad9'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:44:46.630795 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-46de2213-8553-4f28-af93-12bdfba8017e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9448c75844b24f5fb671d96d6164fad9'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:44:46.694017 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 851/1738] 10.222.0.89 () {58 vars in 1224 bytes} [Thu Jul 28 17:44:46 2022] DELETE /identity/v3/users/9448c75844b24f5fb671d96d6164fad9 => generated 0 bytes in 597 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:44:46.695352 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46de2213-8553-4f28-af93-12bdfba8017e admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:46.702748 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-46de2213-8553-4f28-af93-12bdfba8017e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:46.740696 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a55e89-49f2-4e0a-9b43-41f804de8e79 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:46.991834 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46de2213-8553-4f28-af93-12bdfba8017e admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:47.021433 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9412fb63-77a3-4b16-a52b-f1113c1b858d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:47.022163 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9412fb63-77a3-4b16-a52b-f1113c1b858d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:47.022449 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9412fb63-77a3-4b16-a52b-f1113c1b858d admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:47.022824 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9412fb63-77a3-4b16-a52b-f1113c1b858d admin admin] PATH_INFO: `/v3/users/93fc422986f64994bf8eb865443c7c6c` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:47.024723 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9412fb63-77a3-4b16-a52b-f1113c1b858d admin admin] RBAC: Authorizing `identity:delete_user(user_id=93fc422986f64994bf8eb865443c7c6c)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:47.026768 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9412fb63-77a3-4b16-a52b-f1113c1b858d admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:47.041081 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931b2987-d0ad-45f6-aed9-cabfca27a014 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd10b49b29df94404a71f43273839c252', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:47.041877 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931b2987-d0ad-45f6-aed9-cabfca27a014 service nova] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:47.042167 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931b2987-d0ad-45f6-aed9-cabfca27a014 service nova] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:47.042438 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931b2987-d0ad-45f6-aed9-cabfca27a014 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:47.053319 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-931b2987-d0ad-45f6-aed9-cabfca27a014 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:47.188580 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-9412fb63-77a3-4b16-a52b-f1113c1b858d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93fc422986f64994bf8eb865443c7c6c'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:44:47.206444 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-9412fb63-77a3-4b16-a52b-f1113c1b858d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93fc422986f64994bf8eb865443c7c6c'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:44:47.230429 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-9412fb63-77a3-4b16-a52b-f1113c1b858d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93fc422986f64994bf8eb865443c7c6c'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:44:47.231367 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-9412fb63-77a3-4b16-a52b-f1113c1b858d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93fc422986f64994bf8eb865443c7c6c'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:44:47.232733 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-9412fb63-77a3-4b16-a52b-f1113c1b858d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93fc422986f64994bf8eb865443c7c6c'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:44:47.281605 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931b2987-d0ad-45f6-aed9-cabfca27a014 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:47.283269 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931b2987-d0ad-45f6-aed9-cabfca27a014 service nova] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:47.291936 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 852/1739] 10.222.0.89 () {58 vars in 1224 bytes} [Thu Jul 28 17:44:46 2022] DELETE /identity/v3/users/93fc422986f64994bf8eb865443c7c6c => generated 0 bytes in 598 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:44:47.297903 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0745a4-022a-43ae-b7b4-3324e63f87c0 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:47.299618 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0745a4-022a-43ae-b7b4-3324e63f87c0 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:47.300007 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0745a4-022a-43ae-b7b4-3324e63f87c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:47.310165 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 888/1740] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:44:46 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 845 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:44:47.319077 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931b2987-d0ad-45f6-aed9-cabfca27a014 service nova] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:47.320659 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-931b2987-d0ad-45f6-aed9-cabfca27a014 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:47.370754 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-0c0745a4-022a-43ae-b7b4-3324e63f87c0 None None] MFA Rules not processed for user `6bfb0add8a7248e692f177ae58d782e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:44:47.374913 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-0c0745a4-022a-43ae-b7b4-3324e63f87c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:47.415292 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:44:47.487982 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:44:47.572781 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 853/1741] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:44:47 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:44:47.573992 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0745a4-022a-43ae-b7b4-3324e63f87c0 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:47.575858 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-0c0745a4-022a-43ae-b7b4-3324e63f87c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:47.658222 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931b2987-d0ad-45f6-aed9-cabfca27a014 service nova] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:47.779004 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0745a4-022a-43ae-b7b4-3324e63f87c0 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:47.803705 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d76a191-ae6d-4266-9fdb-1c50e8bdebcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:47.804659 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d76a191-ae6d-4266-9fdb-1c50e8bdebcb admin admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:47.805145 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d76a191-ae6d-4266-9fdb-1c50e8bdebcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:47.805672 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d76a191-ae6d-4266-9fdb-1c50e8bdebcb admin admin] PATH_INFO: `/v3/projects/4ead0f9dfd1348689fda7cf01830ecef/users/8cec37b504764378b20d9610287364d7/roles/11f8731756fe4caca13efa61b71b11c5` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:47.825952 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d76a191-ae6d-4266-9fdb-1c50e8bdebcb admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ead0f9dfd1348689fda7cf01830ecef, user_id=8cec37b504764378b20d9610287364d7, role_id=11f8731756fe4caca13efa61b71b11c5)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:47.835786 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d76a191-ae6d-4266-9fdb-1c50e8bdebcb admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:47.854357 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5bd8d5-4965-404e-bb2d-98ef426d1f28 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c59473b01bf4433ba3b111114fc048b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:47.854993 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5bd8d5-4965-404e-bb2d-98ef426d1f28 service swift] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:47.855276 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5bd8d5-4965-404e-bb2d-98ef426d1f28 service swift] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:47.855549 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5bd8d5-4965-404e-bb2d-98ef426d1f28 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:47.857462 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-7d5bd8d5-4965-404e-bb2d-98ef426d1f28 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:47.862907 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 854/1742] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:44:47 2022] PUT /identity/v3/projects/4ead0f9dfd1348689fda7cf01830ecef/users/8cec37b504764378b20d9610287364d7/roles/11f8731756fe4caca13efa61b71b11c5 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:44:47.867398 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f641989-cbde-4b12-951c-72db6ac1e7ef None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:47.867819 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f641989-cbde-4b12-951c-72db6ac1e7ef None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:47.868719 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f641989-cbde-4b12-951c-72db6ac1e7ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:47.921004 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-2f641989-cbde-4b12-951c-72db6ac1e7ef None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:44:47.922447 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-2f641989-cbde-4b12-951c-72db6ac1e7ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:48.014318 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5bd8d5-4965-404e-bb2d-98ef426d1f28 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:48.016148 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5bd8d5-4965-404e-bb2d-98ef426d1f28 service swift] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:48.041375 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 889/1743] 10.222.0.89 () {60 vars in 1384 bytes} [Thu Jul 28 17:44:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 730 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 17:44:48.041955 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5bd8d5-4965-404e-bb2d-98ef426d1f28 service swift] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:48.043414 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-7d5bd8d5-4965-404e-bb2d-98ef426d1f28 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:48.086385 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5bd8d5-4965-404e-bb2d-98ef426d1f28 service swift] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:48.135312 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 855/1744] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:44:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:44:48.136819 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23f6ee8-02e8-4398-9247-5e5eb858790a None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:48.137111 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23f6ee8-02e8-4398-9247-5e5eb858790a None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:48.137416 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23f6ee8-02e8-4398-9247-5e5eb858790a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:48.281747 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-d23f6ee8-02e8-4398-9247-5e5eb858790a None None] MFA Rules not processed for user `8cec37b504764378b20d9610287364d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:44:48.282848 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbaba50-337f-4679-83d3-9e6b3172ffe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:48.283850 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbaba50-337f-4679-83d3-9e6b3172ffe9 admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:48.284381 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbaba50-337f-4679-83d3-9e6b3172ffe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:48.284842 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbaba50-337f-4679-83d3-9e6b3172ffe9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:48.286804 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbaba50-337f-4679-83d3-9e6b3172ffe9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:48.288665 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-d23f6ee8-02e8-4398-9247-5e5eb858790a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:48.291926 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbaba50-337f-4679-83d3-9e6b3172ffe9 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:48.312355 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:44:48.377376 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 890/1745] 10.222.0.89 () {60 vars in 1153 bytes} [Thu Jul 28 17:44:48 2022] POST /identity/v3/projects => generated 371 bytes in 336 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:44:48.378404 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbaba50-337f-4679-83d3-9e6b3172ffe9 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:48.383398 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-6bbaba50-337f-4679-83d3-9e6b3172ffe9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:48.470863 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:44:48.528289 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbaba50-337f-4679-83d3-9e6b3172ffe9 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:48.552139 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd48e67-78f9-4546-ae7d-4a7200bba79b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76817cd541294331b0023069092e8e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:48.552788 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd48e67-78f9-4546-ae7d-4a7200bba79b service neutron] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:48.553129 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd48e67-78f9-4546-ae7d-4a7200bba79b service neutron] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:48.553406 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd48e67-78f9-4546-ae7d-4a7200bba79b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:48.575752 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 856/1746] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:44:48 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 440 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:44:48.576466 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23f6ee8-02e8-4398-9247-5e5eb858790a None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:48.613046 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23f6ee8-02e8-4398-9247-5e5eb858790a None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:48.812205 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f394964-238f-4188-bb70-b8fcf2c76697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:48.813079 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f394964-238f-4188-bb70-b8fcf2c76697 admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:48.813079 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f394964-238f-4188-bb70-b8fcf2c76697 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:48.813244 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f394964-238f-4188-bb70-b8fcf2c76697 admin admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:48.818831 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f394964-238f-4188-bb70-b8fcf2c76697 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:48.826181 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f394964-238f-4188-bb70-b8fcf2c76697 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:48.846515 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd48e67-78f9-4546-ae7d-4a7200bba79b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:48.852071 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd48e67-78f9-4546-ae7d-4a7200bba79b service neutron] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:48.888641 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 857/1747] 10.222.0.89 () {60 vars in 1144 bytes} [Thu Jul 28 17:44:48 2022] POST /identity/v3/users => generated 292 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:44:48.890577 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f394964-238f-4188-bb70-b8fcf2c76697 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:48.893374 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 891/1748] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:44:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 515 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:44:48.898648 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f78bc5-ac2c-4597-aa51-e1ef44d07108 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:48.899255 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f78bc5-ac2c-4597-aa51-e1ef44d07108 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:48.909680 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f78bc5-ac2c-4597-aa51-e1ef44d07108 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:48.916765 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f394964-238f-4188-bb70-b8fcf2c76697 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:48.920788 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 892/1749] 10.222.0.89 () {54 vars in 912 bytes} [Thu Jul 28 17:44:48 2022] GET /identity => generated 270 bytes in 26 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:44:48.922193 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f78bc5-ac2c-4597-aa51-e1ef44d07108 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:48.930564 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03f7db6-710f-431f-bbe4-f17b0bd577fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:48.931201 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03f7db6-710f-431f-bbe4-f17b0bd577fb admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:48.931861 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03f7db6-710f-431f-bbe4-f17b0bd577fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:48.931861 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03f7db6-710f-431f-bbe4-f17b0bd577fb admin admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:48.932641 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03f7db6-710f-431f-bbe4-f17b0bd577fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:48.938613 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03f7db6-710f-431f-bbe4-f17b0bd577fb admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:48.960685 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f78bc5-ac2c-4597-aa51-e1ef44d07108 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:48.979843 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8678a0a-eee8-4468-b154-ed71dab2f6ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:48.980647 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8678a0a-eee8-4468-b154-ed71dab2f6ae admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:48.981048 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8678a0a-eee8-4468-b154-ed71dab2f6ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:48.981423 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8678a0a-eee8-4468-b154-ed71dab2f6ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:48.984097 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8678a0a-eee8-4468-b154-ed71dab2f6ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:48.989557 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8678a0a-eee8-4468-b154-ed71dab2f6ae admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:49.029642 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 893/1750] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:44:48 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:44:49.029642 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a27fcf-af97-4bec-bfdd-4d5dd222e385 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:49.029642 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a27fcf-af97-4bec-bfdd-4d5dd222e385 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:49.030977 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a27fcf-af97-4bec-bfdd-4d5dd222e385 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:49.054377 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 858/1751] 10.222.0.89 () {60 vars in 1153 bytes} [Thu Jul 28 17:44:48 2022] POST /identity/v3/projects => generated 383 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:44:49.054897 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03f7db6-710f-431f-bbe4-f17b0bd577fb admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:49.197814 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-65a27fcf-af97-4bec-bfdd-4d5dd222e385 None None] MFA Rules not processed for user `76817cd541294331b0023069092e8e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:44:49.212239 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-65a27fcf-af97-4bec-bfdd-4d5dd222e385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:49.371539 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03f7db6-710f-431f-bbe4-f17b0bd577fb admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:49.388910 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc66c067-99b7-4808-adf6-a68a0a293b09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:49.389869 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc66c067-99b7-4808-adf6-a68a0a293b09 admin admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:49.390340 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc66c067-99b7-4808-adf6-a68a0a293b09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:49.390628 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc66c067-99b7-4808-adf6-a68a0a293b09 admin admin] PATH_INFO: `/v3/projects/fc80d51e20d5477ca93fb98d6815a048/users/9a144c2042054e4fa67614033924c06d/roles/73e457231a824ea98c28d22610be640c` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:49.460378 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 894/1752] 10.222.0.89 () {58 vars in 1009 bytes} [Thu Jul 28 17:44:49 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 433 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:44:49.461030 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc66c067-99b7-4808-adf6-a68a0a293b09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc80d51e20d5477ca93fb98d6815a048, user_id=9a144c2042054e4fa67614033924c06d, role_id=73e457231a824ea98c28d22610be640c)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:49.462031 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a27fcf-af97-4bec-bfdd-4d5dd222e385 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:49.472089 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc66c067-99b7-4808-adf6-a68a0a293b09 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:49.511554 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 859/1753] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:44:49 2022] PUT /identity/v3/projects/fc80d51e20d5477ca93fb98d6815a048/users/9a144c2042054e4fa67614033924c06d/roles/73e457231a824ea98c28d22610be640c => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:44:49.512941 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a33160c-54e9-461f-babb-d1ef9f323389 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:49.514279 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a27fcf-af97-4bec-bfdd-4d5dd222e385 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:49.515799 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a33160c-54e9-461f-babb-d1ef9f323389 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:49.515930 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a33160c-54e9-461f-babb-d1ef9f323389 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:49.574337 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-7a33160c-54e9-461f-babb-d1ef9f323389 None None] MFA Rules not processed for user `76817cd541294331b0023069092e8e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:44:49.578595 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-7a33160c-54e9-461f-babb-d1ef9f323389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:49.602212 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:44:49.717436 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:44:49.739279 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 860/1754] 10.222.0.100 () {58 vars in 1016 bytes} [Thu Jul 28 17:44:49 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:44:49.742745 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a33160c-54e9-461f-babb-d1ef9f323389 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:49.823480 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c28fb8-9b54-45d0-9721-80bf2f57e9da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:49.828920 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c28fb8-9b54-45d0-9721-80bf2f57e9da admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:49.828920 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c28fb8-9b54-45d0-9721-80bf2f57e9da admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:49.828920 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c28fb8-9b54-45d0-9721-80bf2f57e9da admin admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:49.829960 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c28fb8-9b54-45d0-9721-80bf2f57e9da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:49.832155 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c28fb8-9b54-45d0-9721-80bf2f57e9da admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:49.930917 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 895/1755] 10.222.0.89 () {60 vars in 1144 bytes} [Thu Jul 28 17:44:49 2022] POST /identity/v3/users => generated 298 bytes in 470 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:44:49.932024 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c28fb8-9b54-45d0-9721-80bf2f57e9da admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:49.944570 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-78c28fb8-9b54-45d0-9721-80bf2f57e9da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:49.964832 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a33160c-54e9-461f-babb-d1ef9f323389 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:49.972796 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2efde8-0131-4703-9a0a-e6721f0a078b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:49.973620 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2efde8-0131-4703-9a0a-e6721f0a078b admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:49.973995 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2efde8-0131-4703-9a0a-e6721f0a078b admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:49.974428 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2efde8-0131-4703-9a0a-e6721f0a078b admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:49.975535 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2efde8-0131-4703-9a0a-e6721f0a078b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:49.980553 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2efde8-0131-4703-9a0a-e6721f0a078b admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:49.994749 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c28fb8-9b54-45d0-9721-80bf2f57e9da admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:50.017187 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 861/1756] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:44:49 2022] GET /identity/v3/roles => generated 1420 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:44:50.018711 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2efde8-0131-4703-9a0a-e6721f0a078b admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:50.020309 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483f5d33-cef8-4918-8eac-9dbdad552319 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76817cd541294331b0023069092e8e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:50.021502 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483f5d33-cef8-4918-8eac-9dbdad552319 service neutron] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:50.021879 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483f5d33-cef8-4918-8eac-9dbdad552319 service neutron] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:50.022347 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483f5d33-cef8-4918-8eac-9dbdad552319 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:50.046568 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483f5d33-cef8-4918-8eac-9dbdad552319 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:50.050783 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2efde8-0131-4703-9a0a-e6721f0a078b admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:50.051022 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483f5d33-cef8-4918-8eac-9dbdad552319 service neutron] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:50.060982 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64444644-5d62-4099-8e37-d587adbbe0e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76817cd541294331b0023069092e8e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:50.072806 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64444644-5d62-4099-8e37-d587adbbe0e5 service neutron] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:50.076850 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64444644-5d62-4099-8e37-d587adbbe0e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:50.076850 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64444644-5d62-4099-8e37-d587adbbe0e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:50.084710 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 896/1757] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:44:49 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:44:50.086009 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483f5d33-cef8-4918-8eac-9dbdad552319 service neutron] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:50.097000 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64444644-5d62-4099-8e37-d587adbbe0e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:50.100465 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64444644-5d62-4099-8e37-d587adbbe0e5 service neutron] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:50.120961 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483f5d33-cef8-4918-8eac-9dbdad552319 service neutron] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:50.126760 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 862/1758] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:44:50 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:44:50.129472 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64444644-5d62-4099-8e37-d587adbbe0e5 service neutron] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:50.134725 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5420f85e-3f89-4fc7-b003-b56f769a6981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:50.135515 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5420f85e-3f89-4fc7-b003-b56f769a6981 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:50.135853 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5420f85e-3f89-4fc7-b003-b56f769a6981 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:50.136056 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5420f85e-3f89-4fc7-b003-b56f769a6981 admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:50.136754 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5420f85e-3f89-4fc7-b003-b56f769a6981 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:50.138732 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5420f85e-3f89-4fc7-b003-b56f769a6981 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:50.155962 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64444644-5d62-4099-8e37-d587adbbe0e5 service neutron] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:50.173946 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93951e29-59b7-474e-af38-5ad9a75bba8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:50.174609 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 897/1759] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:44:50 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:44:50.175400 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5420f85e-3f89-4fc7-b003-b56f769a6981 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:50.176199 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93951e29-59b7-474e-af38-5ad9a75bba8f admin admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:50.176479 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93951e29-59b7-474e-af38-5ad9a75bba8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:50.176751 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93951e29-59b7-474e-af38-5ad9a75bba8f admin admin] PATH_INFO: `/v3/projects/fc80d51e20d5477ca93fb98d6815a048/users/9a144c2042054e4fa67614033924c06d/roles/73e457231a824ea98c28d22610be640c` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:50.181987 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93951e29-59b7-474e-af38-5ad9a75bba8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc80d51e20d5477ca93fb98d6815a048, user_id=9a144c2042054e4fa67614033924c06d, role_id=73e457231a824ea98c28d22610be640c)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:50.192840 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93951e29-59b7-474e-af38-5ad9a75bba8f admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:50.201080 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5420f85e-3f89-4fc7-b003-b56f769a6981 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:50.223978 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a794d0-3653-4b52-ab18-4bb2182c5f0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:50.226097 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a794d0-3653-4b52-ab18-4bb2182c5f0f admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:50.226393 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a794d0-3653-4b52-ab18-4bb2182c5f0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:50.226675 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a794d0-3653-4b52-ab18-4bb2182c5f0f admin admin] PATH_INFO: `/v3/projects/0cb535a42e2f442bb36e435350c0cee0/users/4a00baffb5c4461daf07945aa04c635f/roles/73e457231a824ea98c28d22610be640c` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:50.243575 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 863/1760] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:44:50 2022] PUT /identity/v3/projects/fc80d51e20d5477ca93fb98d6815a048/users/9a144c2042054e4fa67614033924c06d/roles/73e457231a824ea98c28d22610be640c => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:44:50.246917 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6151279-76d3-456b-80cf-9f491ba48e7a None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:50.247200 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6151279-76d3-456b-80cf-9f491ba48e7a None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:50.247478 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6151279-76d3-456b-80cf-9f491ba48e7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:50.291705 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a794d0-3653-4b52-ab18-4bb2182c5f0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=0cb535a42e2f442bb36e435350c0cee0, user_id=4a00baffb5c4461daf07945aa04c635f, role_id=73e457231a824ea98c28d22610be640c)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:50.299184 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a794d0-3653-4b52-ab18-4bb2182c5f0f admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:50.346875 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 898/1761] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:44:50 2022] PUT /identity/v3/projects/0cb535a42e2f442bb36e435350c0cee0/users/4a00baffb5c4461daf07945aa04c635f/roles/73e457231a824ea98c28d22610be640c => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:44:50.353653 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a794d0-3653-4b52-ab18-4bb2182c5f0f admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:50.359127 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-f6151279-76d3-456b-80cf-9f491ba48e7a None None] MFA Rules not processed for user `9a144c2042054e4fa67614033924c06d`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:44:50.365784 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-f6151279-76d3-456b-80cf-9f491ba48e7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:50.568792 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a794d0-3653-4b52-ab18-4bb2182c5f0f admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:50.579451 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f888af24-de8a-4046-ab5d-d1fe8b3f2f15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:50.581444 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f888af24-de8a-4046-ab5d-d1fe8b3f2f15 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:50.582758 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f888af24-de8a-4046-ab5d-d1fe8b3f2f15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:50.584392 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f888af24-de8a-4046-ab5d-d1fe8b3f2f15 admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:50.588054 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f888af24-de8a-4046-ab5d-d1fe8b3f2f15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:50.592258 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f888af24-de8a-4046-ab5d-d1fe8b3f2f15 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:50.631655 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 899/1762] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:44:50 2022] GET /identity/v3/roles => generated 1420 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:44:50.636835 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f888af24-de8a-4046-ab5d-d1fe8b3f2f15 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:50.646298 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 864/1763] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:44:50 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 402 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:44:50.670848 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6151279-76d3-456b-80cf-9f491ba48e7a None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:50.674705 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-f6151279-76d3-456b-80cf-9f491ba48e7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:50.702933 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f888af24-de8a-4046-ab5d-d1fe8b3f2f15 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:50.731525 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c34c6d1-3a9f-4bd2-b2fa-e6a0a60650f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:50.732233 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c34c6d1-3a9f-4bd2-b2fa-e6a0a60650f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:50.732521 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c34c6d1-3a9f-4bd2-b2fa-e6a0a60650f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:50.732801 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c34c6d1-3a9f-4bd2-b2fa-e6a0a60650f2 admin admin] PATH_INFO: `/v3/projects/0cb535a42e2f442bb36e435350c0cee0/users/4a00baffb5c4461daf07945aa04c635f/roles/73e457231a824ea98c28d22610be640c` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:50.741872 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c34c6d1-3a9f-4bd2-b2fa-e6a0a60650f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0cb535a42e2f442bb36e435350c0cee0, user_id=4a00baffb5c4461daf07945aa04c635f, role_id=73e457231a824ea98c28d22610be640c)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:50.745204 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c34c6d1-3a9f-4bd2-b2fa-e6a0a60650f2 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:50.817192 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 900/1764] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:44:50 2022] PUT /identity/v3/projects/0cb535a42e2f442bb36e435350c0cee0/users/4a00baffb5c4461daf07945aa04c635f/roles/73e457231a824ea98c28d22610be640c => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:44:50.832095 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0174d7-abda-49ae-a294-dfc70193de25 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:50.832399 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0174d7-abda-49ae-a294-dfc70193de25 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:50.832678 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0174d7-abda-49ae-a294-dfc70193de25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:50.840812 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:44:50.965329 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6151279-76d3-456b-80cf-9f491ba48e7a None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:50.974589 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-0e0174d7-abda-49ae-a294-dfc70193de25 None None] MFA Rules not processed for user `4a00baffb5c4461daf07945aa04c635f`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:44:50.984489 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-0e0174d7-abda-49ae-a294-dfc70193de25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:51.117882 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67646ba1-3ea7-4a4a-92c6-f4caf17a8dd4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96d512d0c84240918dae6c4a3e8b7b79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:51.119995 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67646ba1-3ea7-4a4a-92c6-f4caf17a8dd4 service cinder] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:51.121681 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67646ba1-3ea7-4a4a-92c6-f4caf17a8dd4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:51.122070 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67646ba1-3ea7-4a4a-92c6-f4caf17a8dd4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:51.146799 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:44:51.221232 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 901/1765] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:44:50 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 386 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:44:51.221232 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9c8abd-8117-445d-8a20-fdebbb3370eb None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:51.221232 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9c8abd-8117-445d-8a20-fdebbb3370eb None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:51.221232 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9c8abd-8117-445d-8a20-fdebbb3370eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:51.283187 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-cc9c8abd-8117-445d-8a20-fdebbb3370eb None None] MFA Rules not processed for user `22479a66b0f34038b144f4def1aa3f43`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:44:51.285683 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-cc9c8abd-8117-445d-8a20-fdebbb3370eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:51.326081 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67646ba1-3ea7-4a4a-92c6-f4caf17a8dd4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:51.329341 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67646ba1-3ea7-4a4a-92c6-f4caf17a8dd4 service cinder] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:51.380260 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 865/1766] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:44:50 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 710 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 17:44:51.381300 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67646ba1-3ea7-4a4a-92c6-f4caf17a8dd4 service cinder] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:51.385017 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-67646ba1-3ea7-4a4a-92c6-f4caf17a8dd4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:51.411063 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67646ba1-3ea7-4a4a-92c6-f4caf17a8dd4 service cinder] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:51.428957 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cca868-084b-4759-8480-800c9a3ed40c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96d512d0c84240918dae6c4a3e8b7b79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:51.429640 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cca868-084b-4759-8480-800c9a3ed40c service cinder] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:51.429921 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cca868-084b-4759-8480-800c9a3ed40c service cinder] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:51.430193 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cca868-084b-4759-8480-800c9a3ed40c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:51.467316 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cca868-084b-4759-8480-800c9a3ed40c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:51.485974 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 902/1767] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:44:51 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:44:51.486189 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cca868-084b-4759-8480-800c9a3ed40c service cinder] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:51.535303 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 866/1768] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:44:51 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:44:51.555426 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9c8abd-8117-445d-8a20-fdebbb3370eb None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:51.581224 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9c8abd-8117-445d-8a20-fdebbb3370eb None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:51.589545 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cca868-084b-4759-8480-800c9a3ed40c service cinder] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:51.592618 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-c7cca868-084b-4759-8480-800c9a3ed40c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:51.600012 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5901df74-800a-4283-b4ec-5cfa73fdf8eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96d512d0c84240918dae6c4a3e8b7b79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:51.604571 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5901df74-800a-4283-b4ec-5cfa73fdf8eb service cinder] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:51.606835 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5901df74-800a-4283-b4ec-5cfa73fdf8eb service cinder] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:51.608126 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5901df74-800a-4283-b4ec-5cfa73fdf8eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:51.861217 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5901df74-800a-4283-b4ec-5cfa73fdf8eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:51.864478 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5901df74-800a-4283-b4ec-5cfa73fdf8eb service cinder] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:51.866054 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cca868-084b-4759-8480-800c9a3ed40c service cinder] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:51.880223 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8ba1b9-8281-41a3-aa25-b6a9f02c0313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:51.882895 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8ba1b9-8281-41a3-aa25-b6a9f02c0313 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:51.883337 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8ba1b9-8281-41a3-aa25-b6a9f02c0313 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:51.884402 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8ba1b9-8281-41a3-aa25-b6a9f02c0313 admin admin] PATH_INFO: `/v3/projects/cc0bd418e02f4f4cb1d7ccec3fc7cda9` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:51.887471 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8ba1b9-8281-41a3-aa25-b6a9f02c0313 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc0bd418e02f4f4cb1d7ccec3fc7cda9)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:51.891574 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8ba1b9-8281-41a3-aa25-b6a9f02c0313 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:51.892644 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 903/1769] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:44:51 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:44:51.894248 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5901df74-800a-4283-b4ec-5cfa73fdf8eb service cinder] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:51.926179 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5901df74-800a-4283-b4ec-5cfa73fdf8eb service cinder] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:51.938862 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d763ff1-64c4-4653-8c0d-de5ebeb9a0f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96d512d0c84240918dae6c4a3e8b7b79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:51.941465 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d763ff1-64c4-4653-8c0d-de5ebeb9a0f7 service cinder] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:51.942838 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d763ff1-64c4-4653-8c0d-de5ebeb9a0f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:51.943717 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d763ff1-64c4-4653-8c0d-de5ebeb9a0f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:51.993483 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:44:52.082981 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-5a8ba1b9-8281-41a3-aa25-b6a9f02c0313 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc0bd418e02f4f4cb1d7ccec3fc7cda9', 'request_id': 'req-5a8ba1b9-8281-41a3-aa25-b6a9f02c0313'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:44:52.106018 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-5a8ba1b9-8281-41a3-aa25-b6a9f02c0313 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc0bd418e02f4f4cb1d7ccec3fc7cda9', 'request_id': 'req-5a8ba1b9-8281-41a3-aa25-b6a9f02c0313'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:44:52.131098 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-5a8ba1b9-8281-41a3-aa25-b6a9f02c0313 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc0bd418e02f4f4cb1d7ccec3fc7cda9', 'request_id': 'req-5a8ba1b9-8281-41a3-aa25-b6a9f02c0313'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:44:52.176553 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-5a8ba1b9-8281-41a3-aa25-b6a9f02c0313 admin admin] The token cache is being invalidate because project cc0bd418e02f4f4cb1d7ccec3fc7cda9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80325) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 17:44:52.176553 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-5a8ba1b9-8281-41a3-aa25-b6a9f02c0313 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:44:52.177803 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-5a8ba1b9-8281-41a3-aa25-b6a9f02c0313 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:44:52.180837 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 867/1770] 10.222.0.89 () {58 vars in 1233 bytes} [Thu Jul 28 17:44:51 2022] DELETE /identity/v3/projects/cc0bd418e02f4f4cb1d7ccec3fc7cda9 => generated 0 bytes in 592 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:44:52.188659 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9b3a33-f783-415c-b53e-ccbd17b82a88 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:52.189225 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9b3a33-f783-415c-b53e-ccbd17b82a88 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:52.190050 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9b3a33-f783-415c-b53e-ccbd17b82a88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:52.248658 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d763ff1-64c4-4653-8c0d-de5ebeb9a0f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:52.251469 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d763ff1-64c4-4653-8c0d-de5ebeb9a0f7 service cinder] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:52.292174 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-ca9b3a33-f783-415c-b53e-ccbd17b82a88 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:44:52.293550 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-ca9b3a33-f783-415c-b53e-ccbd17b82a88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:52.296222 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 904/1771] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:44:51 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:44:52.387586 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d763ff1-64c4-4653-8c0d-de5ebeb9a0f7 service cinder] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:52.389553 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-3d763ff1-64c4-4653-8c0d-de5ebeb9a0f7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:52.452029 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d763ff1-64c4-4653-8c0d-de5ebeb9a0f7 service cinder] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:52.533141 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:44:52.609514 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 868/1772] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:44:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 417 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 17:44:52.611058 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d88be5d-920e-4dfd-9d8d-d0f258fb9ef1 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:52.611348 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d88be5d-920e-4dfd-9d8d-d0f258fb9ef1 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:52.611656 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d88be5d-920e-4dfd-9d8d-d0f258fb9ef1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:52.627209 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d683330-88d8-4c87-af6d-e71eda67a8e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:52.629224 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d683330-88d8-4c87-af6d-e71eda67a8e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:52.629526 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d683330-88d8-4c87-af6d-e71eda67a8e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:52.629778 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d683330-88d8-4c87-af6d-e71eda67a8e0 admin admin] PATH_INFO: `/v3/projects/e3bc2c9c7d254f65a104e3af382aa2d2` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:52.637960 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d683330-88d8-4c87-af6d-e71eda67a8e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3bc2c9c7d254f65a104e3af382aa2d2)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:52.648022 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d683330-88d8-4c87-af6d-e71eda67a8e0 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:52.654158 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-7d88be5d-920e-4dfd-9d8d-d0f258fb9ef1 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:44:52.656568 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-7d88be5d-920e-4dfd-9d8d-d0f258fb9ef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:52.668042 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 869/1773] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:44:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:44:52.674833 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d88be5d-920e-4dfd-9d8d-d0f258fb9ef1 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:52.688681 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d88be5d-920e-4dfd-9d8d-d0f258fb9ef1 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:52.702921 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0503f539-978e-4828-b1d8-f1817a33abde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:52.703953 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0503f539-978e-4828-b1d8-f1817a33abde admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:52.704453 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0503f539-978e-4828-b1d8-f1817a33abde admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:52.704933 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0503f539-978e-4828-b1d8-f1817a33abde admin admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:52.706173 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0503f539-978e-4828-b1d8-f1817a33abde admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:44:52.706947 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0503f539-978e-4828-b1d8-f1817a33abde admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:52.713285 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0503f539-978e-4828-b1d8-f1817a33abde admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:52.753057 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 870/1774] 10.222.0.89 () {58 vars in 1153 bytes} [Thu Jul 28 17:44:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:44:52.757848 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0503f539-978e-4828-b1d8-f1817a33abde admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:52.789247 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0503f539-978e-4828-b1d8-f1817a33abde admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:52.842310 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-8d683330-88d8-4c87-af6d-e71eda67a8e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3bc2c9c7d254f65a104e3af382aa2d2', 'request_id': 'req-8d683330-88d8-4c87-af6d-e71eda67a8e0'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:44:52.864054 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-8d683330-88d8-4c87-af6d-e71eda67a8e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3bc2c9c7d254f65a104e3af382aa2d2', 'request_id': 'req-8d683330-88d8-4c87-af6d-e71eda67a8e0'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:44:52.909215 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-8d683330-88d8-4c87-af6d-e71eda67a8e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3bc2c9c7d254f65a104e3af382aa2d2', 'request_id': 'req-8d683330-88d8-4c87-af6d-e71eda67a8e0'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:44:52.939021 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-8d683330-88d8-4c87-af6d-e71eda67a8e0 admin admin] The token cache is being invalidate because project e3bc2c9c7d254f65a104e3af382aa2d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80324) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 17:44:52.939556 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-8d683330-88d8-4c87-af6d-e71eda67a8e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:44:52.940530 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-8d683330-88d8-4c87-af6d-e71eda67a8e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:44:52.942917 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 905/1775] 10.222.0.89 () {58 vars in 1233 bytes} [Thu Jul 28 17:44:52 2022] DELETE /identity/v3/projects/e3bc2c9c7d254f65a104e3af382aa2d2 => generated 0 bytes in 556 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 17:44:52.962948 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d683330-88d8-4c87-af6d-e71eda67a8e0 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:52.966460 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-8d683330-88d8-4c87-af6d-e71eda67a8e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:52.978171 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b879cbdd-e683-4eca-8de1-f947ea955b4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:52.980481 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b879cbdd-e683-4eca-8de1-f947ea955b4e admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:52.982610 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b879cbdd-e683-4eca-8de1-f947ea955b4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:52.983122 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b879cbdd-e683-4eca-8de1-f947ea955b4e admin admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:52.989229 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b879cbdd-e683-4eca-8de1-f947ea955b4e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:52.996645 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b879cbdd-e683-4eca-8de1-f947ea955b4e admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:53.024253 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d683330-88d8-4c87-af6d-e71eda67a8e0 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:53.071259 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c23c2f-3eea-493f-88c0-0e35ecade05f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:53.072687 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c23c2f-3eea-493f-88c0-0e35ecade05f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:53.073113 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c23c2f-3eea-493f-88c0-0e35ecade05f admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:53.073696 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c23c2f-3eea-493f-88c0-0e35ecade05f admin admin] PATH_INFO: `/v3/users/83c6c3493f954c94abe2e7c3223d5974` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:53.076146 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c23c2f-3eea-493f-88c0-0e35ecade05f admin admin] RBAC: Authorizing `identity:delete_user(user_id=83c6c3493f954c94abe2e7c3223d5974)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:53.079316 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c23c2f-3eea-493f-88c0-0e35ecade05f admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:53.235030 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 871/1776] 10.222.0.89 () {60 vars in 1153 bytes} [Thu Jul 28 17:44:52 2022] POST /identity/v3/projects => generated 365 bytes in 478 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:44:53.250116 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b879cbdd-e683-4eca-8de1-f947ea955b4e admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:53.257990 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-b879cbdd-e683-4eca-8de1-f947ea955b4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:53.289497 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-e1c23c2f-3eea-493f-88c0-0e35ecade05f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83c6c3493f954c94abe2e7c3223d5974'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:44:53.348489 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:44:53.353973 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-e1c23c2f-3eea-493f-88c0-0e35ecade05f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83c6c3493f954c94abe2e7c3223d5974'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:44:53.355686 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-e1c23c2f-3eea-493f-88c0-0e35ecade05f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83c6c3493f954c94abe2e7c3223d5974'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:44:53.389072 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-e1c23c2f-3eea-493f-88c0-0e35ecade05f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83c6c3493f954c94abe2e7c3223d5974'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:44:53.469010 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-e1c23c2f-3eea-493f-88c0-0e35ecade05f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83c6c3493f954c94abe2e7c3223d5974'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:44:53.525060 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 906/1777] 10.222.0.89 () {58 vars in 1224 bytes} [Thu Jul 28 17:44:52 2022] DELETE /identity/v3/users/83c6c3493f954c94abe2e7c3223d5974 => generated 0 bytes in 575 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:44:53.623762 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b879cbdd-e683-4eca-8de1-f947ea955b4e admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:54.021185 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeec37f-19c9-46c8-99b8-a9179952f9c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:54.021932 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeec37f-19c9-46c8-99b8-a9179952f9c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:54.022186 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeec37f-19c9-46c8-99b8-a9179952f9c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:54.022385 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeec37f-19c9-46c8-99b8-a9179952f9c6 admin admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:54.023069 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baeec37f-19c9-46c8-99b8-a9179952f9c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:54.025027 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baeec37f-19c9-46c8-99b8-a9179952f9c6 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:54.123141 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 872/1778] 10.222.0.89 () {60 vars in 1144 bytes} [Thu Jul 28 17:44:53 2022] POST /identity/v3/users => generated 283 bytes in 876 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:44:54.124816 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c23c2f-3eea-493f-88c0-0e35ecade05f admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:54.126567 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-e1c23c2f-3eea-493f-88c0-0e35ecade05f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:54.170462 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c23c2f-3eea-493f-88c0-0e35ecade05f admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:54.195884 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aa877c-594d-458a-a276-58550af9d125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:54.196730 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aa877c-594d-458a-a276-58550af9d125 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:54.197954 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aa877c-594d-458a-a276-58550af9d125 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:54.201633 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aa877c-594d-458a-a276-58550af9d125 admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:54.202724 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79aa877c-594d-458a-a276-58550af9d125 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:54.208878 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79aa877c-594d-458a-a276-58550af9d125 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:54.248975 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 907/1779] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:44:54 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:44:54.254650 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeec37f-19c9-46c8-99b8-a9179952f9c6 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:54.283000 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeec37f-19c9-46c8-99b8-a9179952f9c6 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:54.308244 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90df1d62-7401-4658-a1e2-1415b52bf566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:54.311365 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90df1d62-7401-4658-a1e2-1415b52bf566 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:54.313070 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90df1d62-7401-4658-a1e2-1415b52bf566 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:54.314353 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90df1d62-7401-4658-a1e2-1415b52bf566 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:54.316537 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90df1d62-7401-4658-a1e2-1415b52bf566 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:54.326373 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90df1d62-7401-4658-a1e2-1415b52bf566 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:54.373377 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 873/1780] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:44:54 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:44:54.378716 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aa877c-594d-458a-a276-58550af9d125 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:54.392938 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aa877c-594d-458a-a276-58550af9d125 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:54.405102 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a9480a-7dc6-46ad-a89d-86fd7c961607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:54.406325 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a9480a-7dc6-46ad-a89d-86fd7c961607 admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:54.406325 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a9480a-7dc6-46ad-a89d-86fd7c961607 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:54.406570 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a9480a-7dc6-46ad-a89d-86fd7c961607 admin admin] PATH_INFO: `/v3/projects/d995f269855747cfa22b6ddb8f66db71/users/31f8a00531a341c99161aef8f8ea9c9c/roles/11f8731756fe4caca13efa61b71b11c5` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:54.449852 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a9480a-7dc6-46ad-a89d-86fd7c961607 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d995f269855747cfa22b6ddb8f66db71, user_id=31f8a00531a341c99161aef8f8ea9c9c, role_id=11f8731756fe4caca13efa61b71b11c5)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:54.455868 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a9480a-7dc6-46ad-a89d-86fd7c961607 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:54.460353 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:44:54.551547 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 908/1781] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:44:54 2022] PUT /identity/v3/projects/d995f269855747cfa22b6ddb8f66db71/users/31f8a00531a341c99161aef8f8ea9c9c/roles/11f8731756fe4caca13efa61b71b11c5 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:44:54.553294 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27df5a3e-f7cc-47cf-acda-bb3cbdc14962 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:54.553673 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27df5a3e-f7cc-47cf-acda-bb3cbdc14962 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:54.554146 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27df5a3e-f7cc-47cf-acda-bb3cbdc14962 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:54.693520 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-27df5a3e-f7cc-47cf-acda-bb3cbdc14962 None None] MFA Rules not processed for user `31f8a00531a341c99161aef8f8ea9c9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:44:54.698336 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-27df5a3e-f7cc-47cf-acda-bb3cbdc14962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:54.986157 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 874/1782] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:44:54 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 430 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:44:54.987485 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a9480a-7dc6-46ad-a89d-86fd7c961607 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:55.133543 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a9480a-7dc6-46ad-a89d-86fd7c961607 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:55.151485 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa731ab-4d8a-40e2-b152-acbf87578483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:55.152413 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa731ab-4d8a-40e2-b152-acbf87578483 admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:55.152858 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa731ab-4d8a-40e2-b152-acbf87578483 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:55.153389 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa731ab-4d8a-40e2-b152-acbf87578483 admin admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:55.154245 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa731ab-4d8a-40e2-b152-acbf87578483 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:55.162138 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa731ab-4d8a-40e2-b152-acbf87578483 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:55.230263 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 909/1783] 10.222.0.89 () {60 vars in 1153 bytes} [Thu Jul 28 17:44:54 2022] POST /identity/v3/projects => generated 365 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:44:55.232167 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27df5a3e-f7cc-47cf-acda-bb3cbdc14962 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:55.260906 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27df5a3e-f7cc-47cf-acda-bb3cbdc14962 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:55.371104 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:44:55.465229 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0b3263-4f45-43ba-a655-2ee42c72105f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:55.466557 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0b3263-4f45-43ba-a655-2ee42c72105f admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:55.467401 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0b3263-4f45-43ba-a655-2ee42c72105f admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:55.468363 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0b3263-4f45-43ba-a655-2ee42c72105f admin admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:55.470088 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0b3263-4f45-43ba-a655-2ee42c72105f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:55.476476 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0b3263-4f45-43ba-a655-2ee42c72105f admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:55.539325 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa731ab-4d8a-40e2-b152-acbf87578483 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:55.552238 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 875/1784] 10.222.0.89 () {60 vars in 1144 bytes} [Thu Jul 28 17:44:55 2022] POST /identity/v3/users => generated 289 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:44:55.555784 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa731ab-4d8a-40e2-b152-acbf87578483 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:55.572164 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f54073-9f11-41af-88b1-ef45bc6354db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:55.574803 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f54073-9f11-41af-88b1-ef45bc6354db admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:55.576230 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f54073-9f11-41af-88b1-ef45bc6354db admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:55.576643 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f54073-9f11-41af-88b1-ef45bc6354db admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:55.580680 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f54073-9f11-41af-88b1-ef45bc6354db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:55.584835 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f54073-9f11-41af-88b1-ef45bc6354db admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:55.622126 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0b3263-4f45-43ba-a655-2ee42c72105f admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:55.634538 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 910/1785] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:44:55 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:44:55.636163 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f54073-9f11-41af-88b1-ef45bc6354db admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:55.637499 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0b3263-4f45-43ba-a655-2ee42c72105f admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:55.643062 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-83f54073-9f11-41af-88b1-ef45bc6354db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:55.659186 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ca73e6-129e-4e4b-9f08-0b48947c1b79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:55.660388 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ca73e6-129e-4e4b-9f08-0b48947c1b79 admin admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:55.662678 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ca73e6-129e-4e4b-9f08-0b48947c1b79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:55.669381 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ca73e6-129e-4e4b-9f08-0b48947c1b79 admin admin] PATH_INFO: `/v3/projects/e58049119104464e99609cf83730d2d8/users/db543e5ad1064ffab11099ca204ea86f/roles/73e457231a824ea98c28d22610be640c` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:55.707919 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f54073-9f11-41af-88b1-ef45bc6354db admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:55.723062 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c741e377-070a-47ec-a761-be40b3c6008b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:55.724429 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c741e377-070a-47ec-a761-be40b3c6008b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:55.726032 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c741e377-070a-47ec-a761-be40b3c6008b admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:55.727411 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c741e377-070a-47ec-a761-be40b3c6008b admin admin] PATH_INFO: `/v3/projects/26c7ea56f91b405ebd81f2aae73242b7` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:55.731145 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c741e377-070a-47ec-a761-be40b3c6008b admin admin] RBAC: Authorizing `identity:delete_project(project_id=26c7ea56f91b405ebd81f2aae73242b7)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:55.731683 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ca73e6-129e-4e4b-9f08-0b48947c1b79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e58049119104464e99609cf83730d2d8, user_id=db543e5ad1064ffab11099ca204ea86f, role_id=73e457231a824ea98c28d22610be640c)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:55.735270 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c741e377-070a-47ec-a761-be40b3c6008b admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:55.741374 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ca73e6-129e-4e4b-9f08-0b48947c1b79 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:55.787709 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 876/1786] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:44:55 2022] PUT /identity/v3/projects/e58049119104464e99609cf83730d2d8/users/db543e5ad1064ffab11099ca204ea86f/roles/73e457231a824ea98c28d22610be640c => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 17:44:55.789726 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ca73e6-129e-4e4b-9f08-0b48947c1b79 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:55.914122 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-c741e377-070a-47ec-a761-be40b3c6008b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26c7ea56f91b405ebd81f2aae73242b7', 'request_id': 'req-c741e377-070a-47ec-a761-be40b3c6008b'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:44:55.932800 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-c741e377-070a-47ec-a761-be40b3c6008b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26c7ea56f91b405ebd81f2aae73242b7', 'request_id': 'req-c741e377-070a-47ec-a761-be40b3c6008b'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:44:55.948452 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-c741e377-070a-47ec-a761-be40b3c6008b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26c7ea56f91b405ebd81f2aae73242b7', 'request_id': 'req-c741e377-070a-47ec-a761-be40b3c6008b'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:44:55.978923 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-c741e377-070a-47ec-a761-be40b3c6008b admin admin] The token cache is being invalidate because project 26c7ea56f91b405ebd81f2aae73242b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80324) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 17:44:55.979375 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-c741e377-070a-47ec-a761-be40b3c6008b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:44:55.980149 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-c741e377-070a-47ec-a761-be40b3c6008b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:44:55.982672 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 911/1787] 10.222.0.89 () {58 vars in 1233 bytes} [Thu Jul 28 17:44:55 2022] DELETE /identity/v3/projects/26c7ea56f91b405ebd81f2aae73242b7 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:44:55.989140 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8e813b-2bd8-43d6-b5fd-5b170246c3bd None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:55.989140 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8e813b-2bd8-43d6-b5fd-5b170246c3bd None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:55.989140 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8e813b-2bd8-43d6-b5fd-5b170246c3bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:55.996331 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ca73e6-129e-4e4b-9f08-0b48947c1b79 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:56.037038 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-4f8e813b-2bd8-43d6-b5fd-5b170246c3bd None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:44:56.039188 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-4f8e813b-2bd8-43d6-b5fd-5b170246c3bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:56.255861 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a24e5d-bac7-4943-b712-575e3296b1a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:56.257583 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a24e5d-bac7-4943-b712-575e3296b1a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:56.259811 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a24e5d-bac7-4943-b712-575e3296b1a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:56.261421 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a24e5d-bac7-4943-b712-575e3296b1a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:56.263989 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a24e5d-bac7-4943-b712-575e3296b1a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:56.272111 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a24e5d-bac7-4943-b712-575e3296b1a9 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:56.282580 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:INSERT=3,keystone:SELECT=90,keystone:DELETE=7 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:44:56.316100 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 877/1788] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:44:55 2022] GET /identity/v3/roles => generated 1420 bytes in 526 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:44:56.318664 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a24e5d-bac7-4943-b712-575e3296b1a9 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:56.325388 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-b7a24e5d-bac7-4943-b712-575e3296b1a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:56.344676 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 912/1789] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:44:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 357 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:44:56.363608 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3691721b-e840-4808-820d-a899bfb000dc None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:56.364050 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3691721b-e840-4808-820d-a899bfb000dc None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:56.364580 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3691721b-e840-4808-820d-a899bfb000dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:56.406616 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a24e5d-bac7-4943-b712-575e3296b1a9 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:56.429981 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e325985-c8dc-452b-9f5a-5ce3d133f7ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:56.441141 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e325985-c8dc-452b-9f5a-5ce3d133f7ca admin admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:56.441918 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e325985-c8dc-452b-9f5a-5ce3d133f7ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:56.443859 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e325985-c8dc-452b-9f5a-5ce3d133f7ca admin admin] PATH_INFO: `/v3/projects/e58049119104464e99609cf83730d2d8/users/db543e5ad1064ffab11099ca204ea86f/roles/73e457231a824ea98c28d22610be640c` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:56.451489 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e325985-c8dc-452b-9f5a-5ce3d133f7ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=e58049119104464e99609cf83730d2d8, user_id=db543e5ad1064ffab11099ca204ea86f, role_id=73e457231a824ea98c28d22610be640c)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:56.458758 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e325985-c8dc-452b-9f5a-5ce3d133f7ca admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:56.470288 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-3691721b-e840-4808-820d-a899bfb000dc None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:44:56.477973 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-3691721b-e840-4808-820d-a899bfb000dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:56.516906 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 878/1790] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:44:56 2022] PUT /identity/v3/projects/e58049119104464e99609cf83730d2d8/users/db543e5ad1064ffab11099ca204ea86f/roles/73e457231a824ea98c28d22610be640c => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:44:56.521077 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e325985-c8dc-452b-9f5a-5ce3d133f7ca admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:56.527771 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-5e325985-c8dc-452b-9f5a-5ce3d133f7ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:56.647796 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:44:56.753055 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 913/1791] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:44:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 401 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:44:56.756862 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df49e60-1deb-4602-9a70-f2a53016db93 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:56.763213 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df49e60-1deb-4602-9a70-f2a53016db93 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:56.764513 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df49e60-1deb-4602-9a70-f2a53016db93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:56.846251 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e325985-c8dc-452b-9f5a-5ce3d133f7ca admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:56.864675 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5e0333-433a-4af8-9927-184b12281034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:56.869308 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5e0333-433a-4af8-9927-184b12281034 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:56.869639 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5e0333-433a-4af8-9927-184b12281034 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:56.870695 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5e0333-433a-4af8-9927-184b12281034 admin admin] PATH_INFO: `/v3/users/8cec37b504764378b20d9610287364d7` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:56.873352 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5e0333-433a-4af8-9927-184b12281034 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8cec37b504764378b20d9610287364d7)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:56.881698 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5e0333-433a-4af8-9927-184b12281034 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:56.898650 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-0df49e60-1deb-4602-9a70-f2a53016db93 None None] MFA Rules not processed for user `db543e5ad1064ffab11099ca204ea86f`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:44:56.900295 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-0df49e60-1deb-4602-9a70-f2a53016db93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:57.033391 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-da5e0333-433a-4af8-9927-184b12281034 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8cec37b504764378b20d9610287364d7'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:44:57.069682 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-da5e0333-433a-4af8-9927-184b12281034 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8cec37b504764378b20d9610287364d7'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:44:57.095356 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-da5e0333-433a-4af8-9927-184b12281034 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8cec37b504764378b20d9610287364d7'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:44:57.096982 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-da5e0333-433a-4af8-9927-184b12281034 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8cec37b504764378b20d9610287364d7'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:44:57.097987 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-da5e0333-433a-4af8-9927-184b12281034 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8cec37b504764378b20d9610287364d7'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:44:57.178682 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 879/1792] 10.222.0.89 () {58 vars in 1224 bytes} [Thu Jul 28 17:44:56 2022] DELETE /identity/v3/users/8cec37b504764378b20d9610287364d7 => generated 0 bytes in 661 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:44:57.182827 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5e0333-433a-4af8-9927-184b12281034 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:57.190073 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-da5e0333-433a-4af8-9927-184b12281034 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:57.202732 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 914/1793] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:44:56 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 449 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:44:57.204278 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c94d5d-366d-4678-b067-01924adc4f2d None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:57.204570 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c94d5d-366d-4678-b067-01924adc4f2d None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:57.204847 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c94d5d-366d-4678-b067-01924adc4f2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:57.253663 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-09c94d5d-366d-4678-b067-01924adc4f2d None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:44:57.255544 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-09c94d5d-366d-4678-b067-01924adc4f2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:57.402044 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5e0333-433a-4af8-9927-184b12281034 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:57.425776 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 915/1794] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:44:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:44:57.433671 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f664685-5317-4a68-8a45-4fdd61b5a2ac None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:57.433671 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f664685-5317-4a68-8a45-4fdd61b5a2ac None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:57.434652 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333ea6dd-eddd-4591-8cbf-826b01f7cacd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:57.435023 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f664685-5317-4a68-8a45-4fdd61b5a2ac None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:57.435286 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333ea6dd-eddd-4591-8cbf-826b01f7cacd admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:57.435571 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333ea6dd-eddd-4591-8cbf-826b01f7cacd admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:57.435845 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333ea6dd-eddd-4591-8cbf-826b01f7cacd admin admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:57.437233 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 916/1795] 10.222.0.89 () {54 vars in 921 bytes} [Thu Jul 28 17:44:57 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:44:57.437361 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333ea6dd-eddd-4591-8cbf-826b01f7cacd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:44:57.437543 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333ea6dd-eddd-4591-8cbf-826b01f7cacd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:57.440124 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f664685-5317-4a68-8a45-4fdd61b5a2ac None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:57.441627 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333ea6dd-eddd-4591-8cbf-826b01f7cacd admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:57.443345 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-9f664685-5317-4a68-8a45-4fdd61b5a2ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:57.483027 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:44:57.489896 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 880/1796] 10.222.0.89 () {58 vars in 1153 bytes} [Thu Jul 28 17:44:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:44:57.494506 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333ea6dd-eddd-4591-8cbf-826b01f7cacd admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:57.518905 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333ea6dd-eddd-4591-8cbf-826b01f7cacd admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:57.536100 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f305fbd-5850-4e28-baa9-2a5f186430bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:57.537045 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f305fbd-5850-4e28-baa9-2a5f186430bd admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:57.537045 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f305fbd-5850-4e28-baa9-2a5f186430bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:57.537294 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f305fbd-5850-4e28-baa9-2a5f186430bd admin admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:57.537983 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f305fbd-5850-4e28-baa9-2a5f186430bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:57.545185 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f305fbd-5850-4e28-baa9-2a5f186430bd admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:57.557144 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f664685-5317-4a68-8a45-4fdd61b5a2ac None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:57.562194 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de1798f-4b9b-4908-85a4-11cfad9d2574 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96d512d0c84240918dae6c4a3e8b7b79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:57.563520 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de1798f-4b9b-4908-85a4-11cfad9d2574 service cinder] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:57.564133 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de1798f-4b9b-4908-85a4-11cfad9d2574 service cinder] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:57.565961 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de1798f-4b9b-4908-85a4-11cfad9d2574 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:57.666888 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 881/1797] 10.222.0.89 () {60 vars in 1153 bytes} [Thu Jul 28 17:44:57 2022] POST /identity/v3/projects => generated 365 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:44:57.671543 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de1798f-4b9b-4908-85a4-11cfad9d2574 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:57.671752 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdded2be-97d9-499f-928c-e40cf719bf38 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:57.672199 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdded2be-97d9-499f-928c-e40cf719bf38 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:57.672637 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdded2be-97d9-499f-928c-e40cf719bf38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:57.682478 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de1798f-4b9b-4908-85a4-11cfad9d2574 service cinder] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:57.689280 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 917/1798] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:44:57 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:44:57.690418 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de1798f-4b9b-4908-85a4-11cfad9d2574 service cinder] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:57.691807 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-0de1798f-4b9b-4908-85a4-11cfad9d2574 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:57.743409 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-fdded2be-97d9-499f-928c-e40cf719bf38 None None] MFA Rules not processed for user `d10b49b29df94404a71f43273839c252`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:44:57.766507 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-fdded2be-97d9-499f-928c-e40cf719bf38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:57.790429 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de1798f-4b9b-4908-85a4-11cfad9d2574 service cinder] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:57.791283 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:44:57.886878 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfde166b-e005-4fb0-9875-67db87edbecd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76817cd541294331b0023069092e8e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:57.888715 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfde166b-e005-4fb0-9875-67db87edbecd service neutron] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:57.889200 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfde166b-e005-4fb0-9875-67db87edbecd service neutron] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:57.889628 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfde166b-e005-4fb0-9875-67db87edbecd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:57.978448 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:44:58.002458 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 882/1799] 10.222.0.89 () {58 vars in 1018 bytes} [Thu Jul 28 17:44:57 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 334 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:44:58.007321 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1965b21-6e77-4407-be81-73fef1e4e5cc None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:58.008260 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1965b21-6e77-4407-be81-73fef1e4e5cc None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:58.008605 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1965b21-6e77-4407-be81-73fef1e4e5cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:58.029191 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfde166b-e005-4fb0-9875-67db87edbecd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:58.034760 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfde166b-e005-4fb0-9875-67db87edbecd service neutron] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:58.048946 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 918/1800] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:44:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:44:58.052148 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfde166b-e005-4fb0-9875-67db87edbecd service neutron] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:58.069439 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfde166b-e005-4fb0-9875-67db87edbecd service neutron] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:58.077058 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-c1965b21-6e77-4407-be81-73fef1e4e5cc None None] MFA Rules not processed for user `0b97c60c0360466fb878e1e9141e1f3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:44:58.080333 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-c1965b21-6e77-4407-be81-73fef1e4e5cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:58.087098 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd8c148-3a92-4d3b-a5f4-5a9b9408b617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:58.089502 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd8c148-3a92-4d3b-a5f4-5a9b9408b617 admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:58.090163 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd8c148-3a92-4d3b-a5f4-5a9b9408b617 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:58.094384 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd8c148-3a92-4d3b-a5f4-5a9b9408b617 admin admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:58.096082 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd8c148-3a92-4d3b-a5f4-5a9b9408b617 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:58.099937 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd8c148-3a92-4d3b-a5f4-5a9b9408b617 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:58.149617 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 919/1801] 10.222.0.89 () {60 vars in 1144 bytes} [Thu Jul 28 17:44:58 2022] POST /identity/v3/users => generated 283 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:44:58.151544 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd8c148-3a92-4d3b-a5f4-5a9b9408b617 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:58.209217 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 883/1802] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:44:58 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:44:58.210094 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1965b21-6e77-4407-be81-73fef1e4e5cc None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:58.227150 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-c1965b21-6e77-4407-be81-73fef1e4e5cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:58.280005 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1965b21-6e77-4407-be81-73fef1e4e5cc None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:58.288245 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6a5b64-9061-4c9c-b74c-6e656cadf154 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd10b49b29df94404a71f43273839c252', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:58.289020 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6a5b64-9061-4c9c-b74c-6e656cadf154 service nova] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:58.289419 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6a5b64-9061-4c9c-b74c-6e656cadf154 service nova] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:58.289893 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6a5b64-9061-4c9c-b74c-6e656cadf154 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:58.302106 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6a5b64-9061-4c9c-b74c-6e656cadf154 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:58.304151 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6a5b64-9061-4c9c-b74c-6e656cadf154 service nova] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:58.312797 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd8c148-3a92-4d3b-a5f4-5a9b9408b617 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:58.338561 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 884/1803] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:44:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:44:58.340211 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65096cac-c1b9-483a-9618-c5650d8fd25c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96d512d0c84240918dae6c4a3e8b7b79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:58.340438 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6a5b64-9061-4c9c-b74c-6e656cadf154 service nova] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:58.340832 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65096cac-c1b9-483a-9618-c5650d8fd25c service cinder] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:58.341114 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65096cac-c1b9-483a-9618-c5650d8fd25c service cinder] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:58.341422 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65096cac-c1b9-483a-9618-c5650d8fd25c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:58.350764 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-65096cac-c1b9-483a-9618-c5650d8fd25c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:58.380192 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6a5b64-9061-4c9c-b74c-6e656cadf154 service nova] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:58.386644 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1becd98f-976e-497e-a2de-21d52f80a17d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:58.387587 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1becd98f-976e-497e-a2de-21d52f80a17d admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:58.389023 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1becd98f-976e-497e-a2de-21d52f80a17d admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:58.389543 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1becd98f-976e-497e-a2de-21d52f80a17d admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:58.391630 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1becd98f-976e-497e-a2de-21d52f80a17d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:58.394043 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1becd98f-976e-497e-a2de-21d52f80a17d admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:58.413782 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 885/1804] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:44:58 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:44:58.415216 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1becd98f-976e-497e-a2de-21d52f80a17d admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:58.430068 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1becd98f-976e-497e-a2de-21d52f80a17d admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:58.435925 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c7eaca-59a8-45d9-bb83-896e23fb31fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:58.437333 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c7eaca-59a8-45d9-bb83-896e23fb31fc admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:58.437775 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c7eaca-59a8-45d9-bb83-896e23fb31fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:58.438064 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c7eaca-59a8-45d9-bb83-896e23fb31fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:58.439096 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c7eaca-59a8-45d9-bb83-896e23fb31fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:58.441532 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c7eaca-59a8-45d9-bb83-896e23fb31fc admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:58.456757 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 886/1805] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:44:58 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:44:58.461756 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c7eaca-59a8-45d9-bb83-896e23fb31fc admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:58.476552 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c7eaca-59a8-45d9-bb83-896e23fb31fc admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:58.482366 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614145e8-75ae-40a1-8fef-d9c7108ae121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:58.483260 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614145e8-75ae-40a1-8fef-d9c7108ae121 admin admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:58.483847 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614145e8-75ae-40a1-8fef-d9c7108ae121 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:58.485277 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614145e8-75ae-40a1-8fef-d9c7108ae121 admin admin] PATH_INFO: `/v3/projects/46f811ed40ba42e7a5df1309f90b1281/users/3d521428f56a41909f21b4905b188767/roles/11f8731756fe4caca13efa61b71b11c5` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:58.537223 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614145e8-75ae-40a1-8fef-d9c7108ae121 admin admin] RBAC: Authorizing `identity:create_grant(project_id=46f811ed40ba42e7a5df1309f90b1281, user_id=3d521428f56a41909f21b4905b188767, role_id=11f8731756fe4caca13efa61b71b11c5)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:58.537415 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65096cac-c1b9-483a-9618-c5650d8fd25c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:58.538745 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65096cac-c1b9-483a-9618-c5650d8fd25c service cinder] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:58.545402 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614145e8-75ae-40a1-8fef-d9c7108ae121 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:58.556381 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 920/1806] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:44:58 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:44:58.590461 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 887/1807] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:44:58 2022] PUT /identity/v3/projects/46f811ed40ba42e7a5df1309f90b1281/users/3d521428f56a41909f21b4905b188767/roles/11f8731756fe4caca13efa61b71b11c5 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:44:58.601404 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c8efa0-682f-4357-9a13-449a7d54a6a7 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:58.601695 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c8efa0-682f-4357-9a13-449a7d54a6a7 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:58.601955 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c8efa0-682f-4357-9a13-449a7d54a6a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:58.659085 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-54c8efa0-682f-4357-9a13-449a7d54a6a7 None None] MFA Rules not processed for user `3d521428f56a41909f21b4905b188767`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:44:58.660240 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-54c8efa0-682f-4357-9a13-449a7d54a6a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:58.762661 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:44:58.792088 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614145e8-75ae-40a1-8fef-d9c7108ae121 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:58.820295 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 921/1808] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:44:58 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:44:58.956909 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:44:58.989689 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614145e8-75ae-40a1-8fef-d9c7108ae121 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:59.004229 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00b3e51-10d5-4eb2-9ae1-ca3caf7c17dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:59.004859 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00b3e51-10d5-4eb2-9ae1-ca3caf7c17dc admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:59.005316 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00b3e51-10d5-4eb2-9ae1-ca3caf7c17dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:59.005609 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00b3e51-10d5-4eb2-9ae1-ca3caf7c17dc admin admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:59.006439 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00b3e51-10d5-4eb2-9ae1-ca3caf7c17dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:59.010868 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00b3e51-10d5-4eb2-9ae1-ca3caf7c17dc admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:59.097321 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 888/1809] 10.222.0.89 () {60 vars in 1153 bytes} [Thu Jul 28 17:44:58 2022] POST /identity/v3/projects => generated 365 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:44:59.098550 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c8efa0-682f-4357-9a13-449a7d54a6a7 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:59.143953 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c8efa0-682f-4357-9a13-449a7d54a6a7 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:59.238512 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fc4c0a-16c4-49d2-a609-625e8d755584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:59.238512 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fc4c0a-16c4-49d2-a609-625e8d755584 admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:59.238829 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fc4c0a-16c4-49d2-a609-625e8d755584 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:59.238930 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fc4c0a-16c4-49d2-a609-625e8d755584 admin admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:59.239553 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fc4c0a-16c4-49d2-a609-625e8d755584 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:59.241614 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fc4c0a-16c4-49d2-a609-625e8d755584 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:59.255157 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00b3e51-10d5-4eb2-9ae1-ca3caf7c17dc admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:59.261459 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-e00b3e51-10d5-4eb2-9ae1-ca3caf7c17dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:59.286403 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00b3e51-10d5-4eb2-9ae1-ca3caf7c17dc admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:59.293184 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 922/1810] 10.222.0.89 () {60 vars in 1144 bytes} [Thu Jul 28 17:44:59 2022] POST /identity/v3/users => generated 289 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:44:59.295333 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fc4c0a-16c4-49d2-a609-625e8d755584 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:59.296142 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64da89d9-9cbb-4145-bfd4-e9138810b9a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:59.297322 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64da89d9-9cbb-4145-bfd4-e9138810b9a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:59.297714 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64da89d9-9cbb-4145-bfd4-e9138810b9a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:59.298126 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64da89d9-9cbb-4145-bfd4-e9138810b9a8 admin admin] PATH_INFO: `/v3/projects/4ead0f9dfd1348689fda7cf01830ecef` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:59.300917 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64da89d9-9cbb-4145-bfd4-e9138810b9a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ead0f9dfd1348689fda7cf01830ecef)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:59.304365 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64da89d9-9cbb-4145-bfd4-e9138810b9a8 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:59.318165 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fc4c0a-16c4-49d2-a609-625e8d755584 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:59.331801 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b772360-f1e6-422b-847c-97947448a9b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:59.332437 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b772360-f1e6-422b-847c-97947448a9b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:59.332715 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b772360-f1e6-422b-847c-97947448a9b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:59.332976 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b772360-f1e6-422b-847c-97947448a9b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:59.333827 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b772360-f1e6-422b-847c-97947448a9b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:59.335456 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b772360-f1e6-422b-847c-97947448a9b3 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:59.364573 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 923/1811] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:44:59 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:44:59.369026 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b772360-f1e6-422b-847c-97947448a9b3 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:59.393331 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b772360-f1e6-422b-847c-97947448a9b3 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:59.437358 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-64da89d9-9cbb-4145-bfd4-e9138810b9a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ead0f9dfd1348689fda7cf01830ecef', 'request_id': 'req-64da89d9-9cbb-4145-bfd4-e9138810b9a8'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:44:59.438962 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-64da89d9-9cbb-4145-bfd4-e9138810b9a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ead0f9dfd1348689fda7cf01830ecef', 'request_id': 'req-64da89d9-9cbb-4145-bfd4-e9138810b9a8'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:44:59.447359 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-64da89d9-9cbb-4145-bfd4-e9138810b9a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ead0f9dfd1348689fda7cf01830ecef', 'request_id': 'req-64da89d9-9cbb-4145-bfd4-e9138810b9a8'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:44:59.474432 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-64da89d9-9cbb-4145-bfd4-e9138810b9a8 admin admin] The token cache is being invalidate because project 4ead0f9dfd1348689fda7cf01830ecef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80325) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 17:44:59.474846 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-64da89d9-9cbb-4145-bfd4-e9138810b9a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:44:59.475983 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-64da89d9-9cbb-4145-bfd4-e9138810b9a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:44:59.481319 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 889/1812] 10.222.0.89 () {58 vars in 1233 bytes} [Thu Jul 28 17:44:59 2022] DELETE /identity/v3/projects/4ead0f9dfd1348689fda7cf01830ecef => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 17:44:59.482271 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64da89d9-9cbb-4145-bfd4-e9138810b9a8 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:59.485786 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-64da89d9-9cbb-4145-bfd4-e9138810b9a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:59.525909 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a3a494-220a-412a-ba80-514adad98d43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:59.527162 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a3a494-220a-412a-ba80-514adad98d43 admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:59.527451 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a3a494-220a-412a-ba80-514adad98d43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:59.528610 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a3a494-220a-412a-ba80-514adad98d43 admin admin] PATH_INFO: `/v3/projects/12330661b4884565aff5c8b10b9491d8/users/032f95fd5d054796ae57ca884d606a0f/roles/73e457231a824ea98c28d22610be640c` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:59.555982 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a3a494-220a-412a-ba80-514adad98d43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=12330661b4884565aff5c8b10b9491d8, user_id=032f95fd5d054796ae57ca884d606a0f, role_id=73e457231a824ea98c28d22610be640c)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:59.559911 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a3a494-220a-412a-ba80-514adad98d43 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:59.587766 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 924/1813] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:44:59 2022] PUT /identity/v3/projects/12330661b4884565aff5c8b10b9491d8/users/032f95fd5d054796ae57ca884d606a0f/roles/73e457231a824ea98c28d22610be640c => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 17:44:59.588972 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a3a494-220a-412a-ba80-514adad98d43 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:59.593363 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-f4a3a494-220a-412a-ba80-514adad98d43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:44:59.595592 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64da89d9-9cbb-4145-bfd4-e9138810b9a8 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:59.753981 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a3a494-220a-412a-ba80-514adad98d43 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:59.754351 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa77d9f1-9c1e-414f-9831-52417af2cdbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:59.755258 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa77d9f1-9c1e-414f-9831-52417af2cdbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:59.755679 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa77d9f1-9c1e-414f-9831-52417af2cdbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:59.756054 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa77d9f1-9c1e-414f-9831-52417af2cdbe admin admin] PATH_INFO: `/v3/users/4a00baffb5c4461daf07945aa04c635f` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:59.757882 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa77d9f1-9c1e-414f-9831-52417af2cdbe admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a00baffb5c4461daf07945aa04c635f)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:59.760876 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa77d9f1-9c1e-414f-9831-52417af2cdbe admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:59.767208 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95aa0167-d3ce-4c41-b4e7-17d69946be32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:59.768102 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aa0167-d3ce-4c41-b4e7-17d69946be32 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:59.768990 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aa0167-d3ce-4c41-b4e7-17d69946be32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:59.769529 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aa0167-d3ce-4c41-b4e7-17d69946be32 admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:59.770586 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95aa0167-d3ce-4c41-b4e7-17d69946be32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:59.772798 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95aa0167-d3ce-4c41-b4e7-17d69946be32 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:59.782793 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:44:59.801976 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 925/1814] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:44:59 2022] GET /identity/v3/roles => generated 1420 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:44:59.804062 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95aa0167-d3ce-4c41-b4e7-17d69946be32 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:44:59.834760 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95aa0167-d3ce-4c41-b4e7-17d69946be32 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:44:59.837879 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:44:59.840758 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c56e65-1681-4b17-9130-d3247be76d8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:44:59.841487 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c56e65-1681-4b17-9130-d3247be76d8a admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:59.841778 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c56e65-1681-4b17-9130-d3247be76d8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:59.842133 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c56e65-1681-4b17-9130-d3247be76d8a admin admin] PATH_INFO: `/v3/projects/12330661b4884565aff5c8b10b9491d8/users/032f95fd5d054796ae57ca884d606a0f/roles/73e457231a824ea98c28d22610be640c` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:59.844903 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c56e65-1681-4b17-9130-d3247be76d8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=12330661b4884565aff5c8b10b9491d8, user_id=032f95fd5d054796ae57ca884d606a0f, role_id=73e457231a824ea98c28d22610be640c)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:44:59.847877 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c56e65-1681-4b17-9130-d3247be76d8a admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:44:59.880690 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 926/1815] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:44:59 2022] PUT /identity/v3/projects/12330661b4884565aff5c8b10b9491d8/users/032f95fd5d054796ae57ca884d606a0f/roles/73e457231a824ea98c28d22610be640c => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:44:59.893213 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1555288-1dc2-4205-8887-18f5d53052c8 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:44:59.893516 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1555288-1dc2-4205-8887-18f5d53052c8 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:44:59.893795 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1555288-1dc2-4205-8887-18f5d53052c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:44:59.949776 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-c1555288-1dc2-4205-8887-18f5d53052c8 None None] MFA Rules not processed for user `032f95fd5d054796ae57ca884d606a0f`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:44:59.954295 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-c1555288-1dc2-4205-8887-18f5d53052c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:00.001410 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-aa77d9f1-9c1e-414f-9831-52417af2cdbe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a00baffb5c4461daf07945aa04c635f'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:00.028590 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-aa77d9f1-9c1e-414f-9831-52417af2cdbe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a00baffb5c4461daf07945aa04c635f'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:00.045762 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-aa77d9f1-9c1e-414f-9831-52417af2cdbe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a00baffb5c4461daf07945aa04c635f'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:00.046793 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-aa77d9f1-9c1e-414f-9831-52417af2cdbe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a00baffb5c4461daf07945aa04c635f'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:00.047574 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-aa77d9f1-9c1e-414f-9831-52417af2cdbe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a00baffb5c4461daf07945aa04c635f'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:00.074334 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 890/1816] 10.222.0.89 () {58 vars in 1224 bytes} [Thu Jul 28 17:44:59 2022] DELETE /identity/v3/users/4a00baffb5c4461daf07945aa04c635f => generated 0 bytes in 592 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:45:00.316867 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 927/1817] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:44:59 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 421 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:45:00.966956 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa77d9f1-9c1e-414f-9831-52417af2cdbe admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:00.975210 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-aa77d9f1-9c1e-414f-9831-52417af2cdbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:01.242140 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa77d9f1-9c1e-414f-9831-52417af2cdbe admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:01.246401 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb1c076-4b9e-4c52-b925-82960d0908c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96d512d0c84240918dae6c4a3e8b7b79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:01.247012 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb1c076-4b9e-4c52-b925-82960d0908c8 service cinder] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:01.247299 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb1c076-4b9e-4c52-b925-82960d0908c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:01.247575 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb1c076-4b9e-4c52-b925-82960d0908c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:01.416169 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb1c076-4b9e-4c52-b925-82960d0908c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:01.419641 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb1c076-4b9e-4c52-b925-82960d0908c8 service cinder] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:01.480692 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 891/1818] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:45:00 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 514 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:45:02.908311 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1555288-1dc2-4205-8887-18f5d53052c8 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:02.910655 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-c1555288-1dc2-4205-8887-18f5d53052c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:03.056885 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1555288-1dc2-4205-8887-18f5d53052c8 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:03.237213 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f83f4a-5f12-4ed1-a51f-ee2dcfb6da73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:03.239051 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f83f4a-5f12-4ed1-a51f-ee2dcfb6da73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:03.239607 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f83f4a-5f12-4ed1-a51f-ee2dcfb6da73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:03.240692 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f83f4a-5f12-4ed1-a51f-ee2dcfb6da73 admin admin] PATH_INFO: `/v3/users/3d521428f56a41909f21b4905b188767` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:03.243370 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f83f4a-5f12-4ed1-a51f-ee2dcfb6da73 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d521428f56a41909f21b4905b188767)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:03.247308 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f83f4a-5f12-4ed1-a51f-ee2dcfb6da73 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:03.317702 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:45:03.392208 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-e3f83f4a-5f12-4ed1-a51f-ee2dcfb6da73 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d521428f56a41909f21b4905b188767'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:03.415245 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-e3f83f4a-5f12-4ed1-a51f-ee2dcfb6da73 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d521428f56a41909f21b4905b188767'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:03.416103 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-e3f83f4a-5f12-4ed1-a51f-ee2dcfb6da73 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d521428f56a41909f21b4905b188767'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:03.438816 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-e3f83f4a-5f12-4ed1-a51f-ee2dcfb6da73 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d521428f56a41909f21b4905b188767'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:03.471160 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-e3f83f4a-5f12-4ed1-a51f-ee2dcfb6da73 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d521428f56a41909f21b4905b188767'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:03.491402 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 928/1819] 10.222.0.89 () {58 vars in 1224 bytes} [Thu Jul 28 17:45:02 2022] DELETE /identity/v3/users/3d521428f56a41909f21b4905b188767 => generated 0 bytes in 584 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 17:45:03.497207 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451922a7-ee6b-4de2-bfed-0e2d9ee048a2 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:03.498023 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451922a7-ee6b-4de2-bfed-0e2d9ee048a2 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:03.498908 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451922a7-ee6b-4de2-bfed-0e2d9ee048a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:03.623868 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-451922a7-ee6b-4de2-bfed-0e2d9ee048a2 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:45:03.626071 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-451922a7-ee6b-4de2-bfed-0e2d9ee048a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:03.790239 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:45:03.928655 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 892/1820] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:45:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 433 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:45:04.069005 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f83f4a-5f12-4ed1-a51f-ee2dcfb6da73 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:04.072123 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-e3f83f4a-5f12-4ed1-a51f-ee2dcfb6da73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:04.256538 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f83f4a-5f12-4ed1-a51f-ee2dcfb6da73 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:04.270795 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c20c57a-4bf6-4481-9100-7bcff3023304 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76817cd541294331b0023069092e8e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:04.271700 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c20c57a-4bf6-4481-9100-7bcff3023304 service neutron] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:04.272186 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c20c57a-4bf6-4481-9100-7bcff3023304 service neutron] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:04.272610 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c20c57a-4bf6-4481-9100-7bcff3023304 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:04.313376 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c20c57a-4bf6-4481-9100-7bcff3023304 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:04.314967 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c20c57a-4bf6-4481-9100-7bcff3023304 service neutron] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:04.359444 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 929/1821] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:45:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:45:04.425614 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451922a7-ee6b-4de2-bfed-0e2d9ee048a2 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:04.427089 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-451922a7-ee6b-4de2-bfed-0e2d9ee048a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:04.441505 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451922a7-ee6b-4de2-bfed-0e2d9ee048a2 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:04.446942 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3903b9-1b17-4fca-9e5e-d0a6acce3050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:04.447796 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3903b9-1b17-4fca-9e5e-d0a6acce3050 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:04.448114 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3903b9-1b17-4fca-9e5e-d0a6acce3050 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:04.448358 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3903b9-1b17-4fca-9e5e-d0a6acce3050 admin admin] PATH_INFO: `/v3/projects/0cb535a42e2f442bb36e435350c0cee0` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:04.449614 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3903b9-1b17-4fca-9e5e-d0a6acce3050 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0cb535a42e2f442bb36e435350c0cee0)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:04.451619 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3903b9-1b17-4fca-9e5e-d0a6acce3050 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:04.526779 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-ac3903b9-1b17-4fca-9e5e-d0a6acce3050 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0cb535a42e2f442bb36e435350c0cee0', 'request_id': 'req-ac3903b9-1b17-4fca-9e5e-d0a6acce3050'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:04.536923 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-ac3903b9-1b17-4fca-9e5e-d0a6acce3050 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0cb535a42e2f442bb36e435350c0cee0', 'request_id': 'req-ac3903b9-1b17-4fca-9e5e-d0a6acce3050'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:04.543472 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-ac3903b9-1b17-4fca-9e5e-d0a6acce3050 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0cb535a42e2f442bb36e435350c0cee0', 'request_id': 'req-ac3903b9-1b17-4fca-9e5e-d0a6acce3050'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:04.558239 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-ac3903b9-1b17-4fca-9e5e-d0a6acce3050 admin admin] The token cache is being invalidate because project 0cb535a42e2f442bb36e435350c0cee0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80325) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 17:45:04.558764 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-ac3903b9-1b17-4fca-9e5e-d0a6acce3050 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:04.559510 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-ac3903b9-1b17-4fca-9e5e-d0a6acce3050 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:04.561391 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 893/1822] 10.222.0.89 () {58 vars in 1233 bytes} [Thu Jul 28 17:45:04 2022] DELETE /identity/v3/projects/0cb535a42e2f442bb36e435350c0cee0 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:45:04.566268 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1637a44-aa99-49c2-a721-f564bcf9c0f2 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:04.566635 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1637a44-aa99-49c2-a721-f564bcf9c0f2 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:04.567005 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1637a44-aa99-49c2-a721-f564bcf9c0f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:04.589349 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-a1637a44-aa99-49c2-a721-f564bcf9c0f2 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:45:04.590835 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-a1637a44-aa99-49c2-a721-f564bcf9c0f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:04.657293 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8ee79b-efcc-41f9-9978-6ffcbf79cdf8 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:04.657916 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8ee79b-efcc-41f9-9978-6ffcbf79cdf8 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:04.658295 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8ee79b-efcc-41f9-9978-6ffcbf79cdf8 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:04.660573 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 894/1823] 10.222.0.89 () {54 vars in 921 bytes} [Thu Jul 28 17:45:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:45:04.667806 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 930/1824] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:45:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 17:45:04.677391 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f9e76c-2fd2-4393-b1d9-99d3c6afcbd3 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:04.677755 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f9e76c-2fd2-4393-b1d9-99d3c6afcbd3 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:04.678127 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f9e76c-2fd2-4393-b1d9-99d3c6afcbd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:04.697260 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50c62f8-caf9-4b8d-bafa-c66692d82a65 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:04.697487 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50c62f8-caf9-4b8d-bafa-c66692d82a65 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:04.697775 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50c62f8-caf9-4b8d-bafa-c66692d82a65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:04.703424 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-75f9e76c-2fd2-4393-b1d9-99d3c6afcbd3 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:45:04.704995 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-75f9e76c-2fd2-4393-b1d9-99d3c6afcbd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:04.714952 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 895/1825] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:45:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:45:04.721325 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f9e76c-2fd2-4393-b1d9-99d3c6afcbd3 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:04.731337 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-f50c62f8-caf9-4b8d-bafa-c66692d82a65 None None] MFA Rules not processed for user `d10b49b29df94404a71f43273839c252`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:45:04.732638 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-f50c62f8-caf9-4b8d-bafa-c66692d82a65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:04.739025 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f9e76c-2fd2-4393-b1d9-99d3c6afcbd3 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:04.747584 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6429e966-a904-43e9-80d1-7a147f9b7fb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:04.748480 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6429e966-a904-43e9-80d1-7a147f9b7fb9 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:04.751045 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6429e966-a904-43e9-80d1-7a147f9b7fb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:04.751814 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6429e966-a904-43e9-80d1-7a147f9b7fb9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:04.753738 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6429e966-a904-43e9-80d1-7a147f9b7fb9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:45:04.754082 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6429e966-a904-43e9-80d1-7a147f9b7fb9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:04.756503 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6429e966-a904-43e9-80d1-7a147f9b7fb9 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:04.781100 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:45:04.797825 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 896/1826] 10.222.0.89 () {58 vars in 1153 bytes} [Thu Jul 28 17:45:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:45:04.799832 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6429e966-a904-43e9-80d1-7a147f9b7fb9 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:04.820220 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6429e966-a904-43e9-80d1-7a147f9b7fb9 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:04.838247 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e5c77b-8166-4cb7-8c8a-8305d258eb49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:04.838878 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e5c77b-8166-4cb7-8c8a-8305d258eb49 admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:04.839080 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e5c77b-8166-4cb7-8c8a-8305d258eb49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:04.839358 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e5c77b-8166-4cb7-8c8a-8305d258eb49 admin admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:04.840338 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e5c77b-8166-4cb7-8c8a-8305d258eb49 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:04.843459 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e5c77b-8166-4cb7-8c8a-8305d258eb49 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:04.863434 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 931/1827] 10.222.0.89 () {58 vars in 1018 bytes} [Thu Jul 28 17:45:04 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:45:04.888833 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50c62f8-caf9-4b8d-bafa-c66692d82a65 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:04.892523 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-f50c62f8-caf9-4b8d-bafa-c66692d82a65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:04.908522 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 897/1828] 10.222.0.89 () {60 vars in 1153 bytes} [Thu Jul 28 17:45:04 2022] POST /identity/v3/projects => generated 387 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 17:45:04.923172 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50c62f8-caf9-4b8d-bafa-c66692d82a65 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:04.923294 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e5c77b-8166-4cb7-8c8a-8305d258eb49 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:05.025382 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6793b22c-0104-4615-ac0b-11b84956cc50 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd10b49b29df94404a71f43273839c252', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:05.025382 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6793b22c-0104-4615-ac0b-11b84956cc50 service nova] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:05.025382 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6793b22c-0104-4615-ac0b-11b84956cc50 service nova] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:05.025382 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6793b22c-0104-4615-ac0b-11b84956cc50 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:05.035859 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6793b22c-0104-4615-ac0b-11b84956cc50 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:05.037981 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6793b22c-0104-4615-ac0b-11b84956cc50 service nova] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:05.048109 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e5c77b-8166-4cb7-8c8a-8305d258eb49 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:05.051669 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 932/1829] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:45:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 17:45:05.053312 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f089135-9b11-4742-b1c5-92d4148bc0e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:05.053945 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f089135-9b11-4742-b1c5-92d4148bc0e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:05.054229 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f089135-9b11-4742-b1c5-92d4148bc0e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:05.054505 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f089135-9b11-4742-b1c5-92d4148bc0e6 admin admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:05.055181 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f089135-9b11-4742-b1c5-92d4148bc0e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:05.057335 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f089135-9b11-4742-b1c5-92d4148bc0e6 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:05.086391 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 898/1830] 10.222.0.89 () {60 vars in 1144 bytes} [Thu Jul 28 17:45:04 2022] POST /identity/v3/users => generated 294 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:45:05.088753 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6793b22c-0104-4615-ac0b-11b84956cc50 service nova] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:05.101179 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6793b22c-0104-4615-ac0b-11b84956cc50 service nova] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:05.106363 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0debf99c-6fa6-4204-8bf6-3d65a734864c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:05.107178 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0debf99c-6fa6-4204-8bf6-3d65a734864c admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:05.107178 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0debf99c-6fa6-4204-8bf6-3d65a734864c admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:05.107472 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0debf99c-6fa6-4204-8bf6-3d65a734864c admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:05.108242 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0debf99c-6fa6-4204-8bf6-3d65a734864c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:05.110952 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0debf99c-6fa6-4204-8bf6-3d65a734864c admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:05.122608 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 933/1831] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:45:05 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:45:05.129419 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f089135-9b11-4742-b1c5-92d4148bc0e6 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:05.141712 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f089135-9b11-4742-b1c5-92d4148bc0e6 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:05.146727 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6aed6e6-abef-48e1-a7ec-980efc08d5f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:05.147515 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6aed6e6-abef-48e1-a7ec-980efc08d5f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:05.147800 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6aed6e6-abef-48e1-a7ec-980efc08d5f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:05.148080 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6aed6e6-abef-48e1-a7ec-980efc08d5f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:05.148787 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6aed6e6-abef-48e1-a7ec-980efc08d5f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:05.150477 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6aed6e6-abef-48e1-a7ec-980efc08d5f1 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:05.152803 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:45:05.163183 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 899/1832] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:45:05 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:45:05.167483 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0debf99c-6fa6-4204-8bf6-3d65a734864c admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:05.178758 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0debf99c-6fa6-4204-8bf6-3d65a734864c admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:05.183619 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c4c592-3186-46fb-b09b-71402406b459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:05.184309 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c4c592-3186-46fb-b09b-71402406b459 admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:05.184596 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c4c592-3186-46fb-b09b-71402406b459 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:05.184875 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c4c592-3186-46fb-b09b-71402406b459 admin admin] PATH_INFO: `/v3/projects/2576b86cf8834856a824ae174e939069/users/e36b72af7b50465084fbd209d67242e2/roles/11f8731756fe4caca13efa61b71b11c5` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:05.200425 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c4c592-3186-46fb-b09b-71402406b459 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2576b86cf8834856a824ae174e939069, user_id=e36b72af7b50465084fbd209d67242e2, role_id=11f8731756fe4caca13efa61b71b11c5)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:05.203357 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c4c592-3186-46fb-b09b-71402406b459 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:05.219819 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 934/1833] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:45:05 2022] PUT /identity/v3/projects/2576b86cf8834856a824ae174e939069/users/e36b72af7b50465084fbd209d67242e2/roles/11f8731756fe4caca13efa61b71b11c5 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:45:05.229110 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491f1a86-fec3-43d6-9c60-47a872969656 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:05.229514 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491f1a86-fec3-43d6-9c60-47a872969656 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:05.229866 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491f1a86-fec3-43d6-9c60-47a872969656 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:05.246176 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c4c592-3186-46fb-b09b-71402406b459 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:05.247971 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-d5c4c592-3186-46fb-b09b-71402406b459 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:05.264932 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-491f1a86-fec3-43d6-9c60-47a872969656 None None] MFA Rules not processed for user `e36b72af7b50465084fbd209d67242e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:45:05.271581 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-491f1a86-fec3-43d6-9c60-47a872969656 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:05.379501 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c4c592-3186-46fb-b09b-71402406b459 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:05.382374 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 900/1834] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:45:05 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:45:05.384816 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afbd712-c28a-459e-9fb8-cb30bc879cbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:05.385500 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afbd712-c28a-459e-9fb8-cb30bc879cbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:05.386054 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afbd712-c28a-459e-9fb8-cb30bc879cbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:05.386341 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afbd712-c28a-459e-9fb8-cb30bc879cbd admin admin] PATH_INFO: `/v3/projects/46f811ed40ba42e7a5df1309f90b1281` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:05.387434 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491f1a86-fec3-43d6-9c60-47a872969656 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:05.387616 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afbd712-c28a-459e-9fb8-cb30bc879cbd admin admin] RBAC: Authorizing `identity:delete_project(project_id=46f811ed40ba42e7a5df1309f90b1281)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:05.389575 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afbd712-c28a-459e-9fb8-cb30bc879cbd admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:05.423892 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491f1a86-fec3-43d6-9c60-47a872969656 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:05.434136 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:45:05.439734 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6198f20a-9b4c-431a-b208-2ef33c99ae48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:05.440587 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6198f20a-9b4c-431a-b208-2ef33c99ae48 admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:05.440901 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6198f20a-9b4c-431a-b208-2ef33c99ae48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:05.441255 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6198f20a-9b4c-431a-b208-2ef33c99ae48 admin admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:05.442131 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6198f20a-9b4c-431a-b208-2ef33c99ae48 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:05.444527 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6198f20a-9b4c-431a-b208-2ef33c99ae48 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:05.471490 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-4afbd712-c28a-459e-9fb8-cb30bc879cbd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46f811ed40ba42e7a5df1309f90b1281', 'request_id': 'req-4afbd712-c28a-459e-9fb8-cb30bc879cbd'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:05.481193 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 901/1835] 10.222.0.89 () {60 vars in 1153 bytes} [Thu Jul 28 17:45:05 2022] POST /identity/v3/projects => generated 389 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:45:05.485512 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6198f20a-9b4c-431a-b208-2ef33c99ae48 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:05.492022 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-4afbd712-c28a-459e-9fb8-cb30bc879cbd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46f811ed40ba42e7a5df1309f90b1281', 'request_id': 'req-4afbd712-c28a-459e-9fb8-cb30bc879cbd'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:05.499036 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-4afbd712-c28a-459e-9fb8-cb30bc879cbd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46f811ed40ba42e7a5df1309f90b1281', 'request_id': 'req-4afbd712-c28a-459e-9fb8-cb30bc879cbd'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:05.515313 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-4afbd712-c28a-459e-9fb8-cb30bc879cbd admin admin] The token cache is being invalidate because project 46f811ed40ba42e7a5df1309f90b1281 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80324) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 17:45:05.515625 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-4afbd712-c28a-459e-9fb8-cb30bc879cbd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:05.516547 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-4afbd712-c28a-459e-9fb8-cb30bc879cbd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:05.518426 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 935/1836] 10.222.0.89 () {58 vars in 1233 bytes} [Thu Jul 28 17:45:05 2022] DELETE /identity/v3/projects/46f811ed40ba42e7a5df1309f90b1281 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 17:45:05.522320 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afbd712-c28a-459e-9fb8-cb30bc879cbd admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:05.524119 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-4afbd712-c28a-459e-9fb8-cb30bc879cbd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:05.566209 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6198f20a-9b4c-431a-b208-2ef33c99ae48 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:05.571648 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8124003a-026e-48ed-8342-045bda3f854c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:05.572239 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8124003a-026e-48ed-8342-045bda3f854c admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:05.572640 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8124003a-026e-48ed-8342-045bda3f854c admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:05.572822 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8124003a-026e-48ed-8342-045bda3f854c admin admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:05.573490 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8124003a-026e-48ed-8342-045bda3f854c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:05.575421 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8124003a-026e-48ed-8342-045bda3f854c admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:05.607502 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 902/1837] 10.222.0.89 () {60 vars in 1144 bytes} [Thu Jul 28 17:45:05 2022] POST /identity/v3/users => generated 301 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 17:45:05.608403 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8124003a-026e-48ed-8342-045bda3f854c admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:05.610734 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-8124003a-026e-48ed-8342-045bda3f854c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:05.621721 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afbd712-c28a-459e-9fb8-cb30bc879cbd admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:05.627777 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d11511f-70fc-43b8-b77f-260a2e181d93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:05.628096 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d11511f-70fc-43b8-b77f-260a2e181d93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:05.628510 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d11511f-70fc-43b8-b77f-260a2e181d93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:05.629885 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d11511f-70fc-43b8-b77f-260a2e181d93 admin admin] PATH_INFO: `/v3/users/032f95fd5d054796ae57ca884d606a0f` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:05.630236 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d11511f-70fc-43b8-b77f-260a2e181d93 admin admin] RBAC: Authorizing `identity:delete_user(user_id=032f95fd5d054796ae57ca884d606a0f)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:05.632252 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d11511f-70fc-43b8-b77f-260a2e181d93 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:05.640791 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8124003a-026e-48ed-8342-045bda3f854c admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:05.646559 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2484d3-4cc3-46b7-a725-a17ab24a1cc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:05.647480 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2484d3-4cc3-46b7-a725-a17ab24a1cc5 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:05.648483 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2484d3-4cc3-46b7-a725-a17ab24a1cc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:05.649075 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2484d3-4cc3-46b7-a725-a17ab24a1cc5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:05.649985 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2484d3-4cc3-46b7-a725-a17ab24a1cc5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:05.652874 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2484d3-4cc3-46b7-a725-a17ab24a1cc5 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:05.674199 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 903/1838] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:45:05 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:45:05.675709 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2484d3-4cc3-46b7-a725-a17ab24a1cc5 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:05.685913 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2484d3-4cc3-46b7-a725-a17ab24a1cc5 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:05.691512 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d51356a-d0cd-494c-baeb-d7dc4717729c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:05.691670 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-0d11511f-70fc-43b8-b77f-260a2e181d93 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '032f95fd5d054796ae57ca884d606a0f'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:05.692432 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d51356a-d0cd-494c-baeb-d7dc4717729c admin admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:05.692558 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d51356a-d0cd-494c-baeb-d7dc4717729c admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:05.692739 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d51356a-d0cd-494c-baeb-d7dc4717729c admin admin] PATH_INFO: `/v3/projects/c7fd42cc001142279634135b4cb6727c/users/32a305c1dfa2485588b272d9c7d53bcc/roles/73e457231a824ea98c28d22610be640c` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:05.699638 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-0d11511f-70fc-43b8-b77f-260a2e181d93 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '032f95fd5d054796ae57ca884d606a0f'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:05.700508 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-0d11511f-70fc-43b8-b77f-260a2e181d93 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '032f95fd5d054796ae57ca884d606a0f'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:05.710429 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-0d11511f-70fc-43b8-b77f-260a2e181d93 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '032f95fd5d054796ae57ca884d606a0f'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:05.711174 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d51356a-d0cd-494c-baeb-d7dc4717729c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7fd42cc001142279634135b4cb6727c, user_id=32a305c1dfa2485588b272d9c7d53bcc, role_id=73e457231a824ea98c28d22610be640c)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:05.715822 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d51356a-d0cd-494c-baeb-d7dc4717729c admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:05.726765 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-0d11511f-70fc-43b8-b77f-260a2e181d93 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '032f95fd5d054796ae57ca884d606a0f'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:05.734764 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 936/1839] 10.222.0.89 () {58 vars in 1224 bytes} [Thu Jul 28 17:45:05 2022] DELETE /identity/v3/users/032f95fd5d054796ae57ca884d606a0f => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 17:45:05.740436 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 904/1840] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:45:05 2022] PUT /identity/v3/projects/c7fd42cc001142279634135b4cb6727c/users/32a305c1dfa2485588b272d9c7d53bcc/roles/73e457231a824ea98c28d22610be640c => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 17:45:05.744254 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d11511f-70fc-43b8-b77f-260a2e181d93 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:05.745638 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-0d11511f-70fc-43b8-b77f-260a2e181d93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:05.784687 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:45:05.825428 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d11511f-70fc-43b8-b77f-260a2e181d93 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:05.831254 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c07f903-edca-4a29-9da2-5345a74d8ede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:05.832011 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c07f903-edca-4a29-9da2-5345a74d8ede admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:05.832283 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c07f903-edca-4a29-9da2-5345a74d8ede admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:05.832555 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c07f903-edca-4a29-9da2-5345a74d8ede admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:05.833306 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c07f903-edca-4a29-9da2-5345a74d8ede admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:05.834928 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c07f903-edca-4a29-9da2-5345a74d8ede admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:05.846957 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 937/1841] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:45:05 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:45:05.851348 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d51356a-d0cd-494c-baeb-d7dc4717729c admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:05.877508 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:45:05.933019 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d51356a-d0cd-494c-baeb-d7dc4717729c admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:05.939408 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed9996b-6f4b-4ca9-83e7-d2a6467ee903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:05.940090 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed9996b-6f4b-4ca9-83e7-d2a6467ee903 admin admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:05.940370 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed9996b-6f4b-4ca9-83e7-d2a6467ee903 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:05.940644 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed9996b-6f4b-4ca9-83e7-d2a6467ee903 admin admin] PATH_INFO: `/v3/projects/c7fd42cc001142279634135b4cb6727c/users/32a305c1dfa2485588b272d9c7d53bcc/roles/73e457231a824ea98c28d22610be640c` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:05.942738 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed9996b-6f4b-4ca9-83e7-d2a6467ee903 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7fd42cc001142279634135b4cb6727c, user_id=32a305c1dfa2485588b272d9c7d53bcc, role_id=73e457231a824ea98c28d22610be640c)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:05.945481 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed9996b-6f4b-4ca9-83e7-d2a6467ee903 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:05.958466 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 905/1842] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:45:05 2022] PUT /identity/v3/projects/c7fd42cc001142279634135b4cb6727c/users/32a305c1dfa2485588b272d9c7d53bcc/roles/73e457231a824ea98c28d22610be640c => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 17:45:05.969606 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84f0bcf-5b18-445f-914a-d2fd1215ae38 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:05.969896 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84f0bcf-5b18-445f-914a-d2fd1215ae38 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:05.970711 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84f0bcf-5b18-445f-914a-d2fd1215ae38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:06.001221 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-b84f0bcf-5b18-445f-914a-d2fd1215ae38 None None] MFA Rules not processed for user `32a305c1dfa2485588b272d9c7d53bcc`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:45:06.001404 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-b84f0bcf-5b18-445f-914a-d2fd1215ae38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:06.119490 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 938/1843] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:45:05 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:45:06.131578 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed9996b-6f4b-4ca9-83e7-d2a6467ee903 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:06.133645 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-9ed9996b-6f4b-4ca9-83e7-d2a6467ee903 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:06.208315 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed9996b-6f4b-4ca9-83e7-d2a6467ee903 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:06.212149 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e1ae97-b5aa-4a33-8aa2-f73e144f2500 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96d512d0c84240918dae6c4a3e8b7b79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:06.213093 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e1ae97-b5aa-4a33-8aa2-f73e144f2500 service cinder] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:06.213522 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e1ae97-b5aa-4a33-8aa2-f73e144f2500 service cinder] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:06.213788 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e1ae97-b5aa-4a33-8aa2-f73e144f2500 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:06.224554 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e1ae97-b5aa-4a33-8aa2-f73e144f2500 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:06.226075 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e1ae97-b5aa-4a33-8aa2-f73e144f2500 service cinder] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:06.239477 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 906/1844] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:45:06 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:45:06.310925 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84f0bcf-5b18-445f-914a-d2fd1215ae38 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:06.312423 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-b84f0bcf-5b18-445f-914a-d2fd1215ae38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:06.387869 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e1ae97-b5aa-4a33-8aa2-f73e144f2500 service cinder] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:06.393235 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84f0bcf-5b18-445f-914a-d2fd1215ae38 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:06.398587 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc3839e-0fb2-4d93-9719-b53280b8744a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:06.399134 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e1ae97-b5aa-4a33-8aa2-f73e144f2500 service cinder] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:06.399237 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc3839e-0fb2-4d93-9719-b53280b8744a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:06.399558 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc3839e-0fb2-4d93-9719-b53280b8744a admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:06.399837 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc3839e-0fb2-4d93-9719-b53280b8744a admin admin] PATH_INFO: `/v3/projects/12330661b4884565aff5c8b10b9491d8` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:06.401000 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc3839e-0fb2-4d93-9719-b53280b8744a admin admin] RBAC: Authorizing `identity:delete_project(project_id=12330661b4884565aff5c8b10b9491d8)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:06.402938 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc3839e-0fb2-4d93-9719-b53280b8744a admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:06.403897 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86409188-db45-4018-ae0d-3b91f1d40258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:06.404526 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86409188-db45-4018-ae0d-3b91f1d40258 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:06.404804 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86409188-db45-4018-ae0d-3b91f1d40258 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:06.405076 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86409188-db45-4018-ae0d-3b91f1d40258 admin admin] PATH_INFO: `/v3/users/e36b72af7b50465084fbd209d67242e2` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:06.407129 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86409188-db45-4018-ae0d-3b91f1d40258 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e36b72af7b50465084fbd209d67242e2)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:06.409054 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86409188-db45-4018-ae0d-3b91f1d40258 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:06.466000 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-2bc3839e-0fb2-4d93-9719-b53280b8744a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12330661b4884565aff5c8b10b9491d8', 'request_id': 'req-2bc3839e-0fb2-4d93-9719-b53280b8744a'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:06.468651 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-86409188-db45-4018-ae0d-3b91f1d40258 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e36b72af7b50465084fbd209d67242e2'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:06.470980 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:45:06.472895 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-2bc3839e-0fb2-4d93-9719-b53280b8744a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12330661b4884565aff5c8b10b9491d8', 'request_id': 'req-2bc3839e-0fb2-4d93-9719-b53280b8744a'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:06.479784 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-2bc3839e-0fb2-4d93-9719-b53280b8744a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12330661b4884565aff5c8b10b9491d8', 'request_id': 'req-2bc3839e-0fb2-4d93-9719-b53280b8744a'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:06.482716 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-86409188-db45-4018-ae0d-3b91f1d40258 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e36b72af7b50465084fbd209d67242e2'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:06.498148 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-2bc3839e-0fb2-4d93-9719-b53280b8744a admin admin] The token cache is being invalidate because project 12330661b4884565aff5c8b10b9491d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80324) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 17:45:06.498148 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-2bc3839e-0fb2-4d93-9719-b53280b8744a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:06.498148 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-2bc3839e-0fb2-4d93-9719-b53280b8744a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:06.499041 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 939/1845] 10.222.0.89 () {58 vars in 1233 bytes} [Thu Jul 28 17:45:06 2022] DELETE /identity/v3/projects/12330661b4884565aff5c8b10b9491d8 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:45:06.503671 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-86409188-db45-4018-ae0d-3b91f1d40258 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e36b72af7b50465084fbd209d67242e2'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:06.504475 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fddb5ff-2c27-483e-b699-0d9518c50e8c None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:06.504689 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fddb5ff-2c27-483e-b699-0d9518c50e8c None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:06.505071 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fddb5ff-2c27-483e-b699-0d9518c50e8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:06.505291 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-86409188-db45-4018-ae0d-3b91f1d40258 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e36b72af7b50465084fbd209d67242e2'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:06.505945 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-86409188-db45-4018-ae0d-3b91f1d40258 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e36b72af7b50465084fbd209d67242e2'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:06.526916 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-2fddb5ff-2c27-483e-b699-0d9518c50e8c None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:45:06.528270 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-2fddb5ff-2c27-483e-b699-0d9518c50e8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:06.533190 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 907/1846] 10.222.0.89 () {58 vars in 1224 bytes} [Thu Jul 28 17:45:06 2022] DELETE /identity/v3/users/e36b72af7b50465084fbd209d67242e2 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 17:45:06.538417 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388cae33-57a1-4cee-a35b-e74bd2e97174 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:06.538940 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388cae33-57a1-4cee-a35b-e74bd2e97174 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:06.539167 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388cae33-57a1-4cee-a35b-e74bd2e97174 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:06.558199 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-388cae33-57a1-4cee-a35b-e74bd2e97174 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:45:06.559389 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-388cae33-57a1-4cee-a35b-e74bd2e97174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:06.586972 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:45:06.608130 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 940/1847] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:45:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 17:45:06.616422 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3d4731-3802-4941-8ac8-08ec99657ac6 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:06.616722 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3d4731-3802-4941-8ac8-08ec99657ac6 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:06.616983 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3d4731-3802-4941-8ac8-08ec99657ac6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:06.627709 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 908/1848] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:45:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:45:06.637091 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-9a3d4731-3802-4941-8ac8-08ec99657ac6 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:45:06.638282 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388cae33-57a1-4cee-a35b-e74bd2e97174 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:06.638412 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-9a3d4731-3802-4941-8ac8-08ec99657ac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:06.639767 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-388cae33-57a1-4cee-a35b-e74bd2e97174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:06.646944 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 941/1849] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:45:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 17:45:06.651030 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3d4731-3802-4941-8ac8-08ec99657ac6 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:06.660606 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3d4731-3802-4941-8ac8-08ec99657ac6 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:06.665332 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7670f34-624e-4ec0-8505-4e94d5846f34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:06.665930 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7670f34-624e-4ec0-8505-4e94d5846f34 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:06.666210 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7670f34-624e-4ec0-8505-4e94d5846f34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:06.666485 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7670f34-624e-4ec0-8505-4e94d5846f34 admin admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:06.667179 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7670f34-624e-4ec0-8505-4e94d5846f34 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:45:06.667484 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7670f34-624e-4ec0-8505-4e94d5846f34 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:06.669079 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7670f34-624e-4ec0-8505-4e94d5846f34 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:06.681400 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 942/1850] 10.222.0.89 () {58 vars in 1153 bytes} [Thu Jul 28 17:45:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:45:06.685714 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7670f34-624e-4ec0-8505-4e94d5846f34 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:06.695843 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7670f34-624e-4ec0-8505-4e94d5846f34 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:06.699599 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388cae33-57a1-4cee-a35b-e74bd2e97174 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:06.700420 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ed5620-8964-4e0f-a56c-6915e84ec4fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:06.701023 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ed5620-8964-4e0f-a56c-6915e84ec4fb admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:06.701341 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ed5620-8964-4e0f-a56c-6915e84ec4fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:06.701616 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ed5620-8964-4e0f-a56c-6915e84ec4fb admin admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:06.702861 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ed5620-8964-4e0f-a56c-6915e84ec4fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:06.704536 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601f8eaf-fc42-4e6b-9ae3-5f3e58679678 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76817cd541294331b0023069092e8e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:06.704778 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ed5620-8964-4e0f-a56c-6915e84ec4fb admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:06.705357 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601f8eaf-fc42-4e6b-9ae3-5f3e58679678 service neutron] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:06.705727 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601f8eaf-fc42-4e6b-9ae3-5f3e58679678 service neutron] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:06.706072 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601f8eaf-fc42-4e6b-9ae3-5f3e58679678 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:06.715609 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601f8eaf-fc42-4e6b-9ae3-5f3e58679678 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:06.717038 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601f8eaf-fc42-4e6b-9ae3-5f3e58679678 service neutron] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:06.727367 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 909/1851] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:45:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:45:06.735784 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 943/1852] 10.222.0.89 () {60 vars in 1153 bytes} [Thu Jul 28 17:45:06 2022] POST /identity/v3/projects => generated 365 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 17:45:06.738266 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601f8eaf-fc42-4e6b-9ae3-5f3e58679678 service neutron] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:06.749809 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601f8eaf-fc42-4e6b-9ae3-5f3e58679678 service neutron] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:06.845850 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed44c0e1-c493-4f4d-8177-49a30f4d73e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:06.846870 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed44c0e1-c493-4f4d-8177-49a30f4d73e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:06.847158 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed44c0e1-c493-4f4d-8177-49a30f4d73e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:06.847440 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed44c0e1-c493-4f4d-8177-49a30f4d73e2 admin admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:06.848130 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed44c0e1-c493-4f4d-8177-49a30f4d73e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:06.851120 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed44c0e1-c493-4f4d-8177-49a30f4d73e2 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:06.914646 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 910/1853] 10.222.0.89 () {60 vars in 1144 bytes} [Thu Jul 28 17:45:06 2022] POST /identity/v3/users => generated 283 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:45:06.929713 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ed5620-8964-4e0f-a56c-6915e84ec4fb admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:06.947626 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ed5620-8964-4e0f-a56c-6915e84ec4fb admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:06.960413 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636587c7-2e2c-4ea7-bd84-adafa9a58d47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:06.961044 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636587c7-2e2c-4ea7-bd84-adafa9a58d47 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:06.961417 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636587c7-2e2c-4ea7-bd84-adafa9a58d47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:06.961692 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636587c7-2e2c-4ea7-bd84-adafa9a58d47 admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:06.962395 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636587c7-2e2c-4ea7-bd84-adafa9a58d47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:06.964256 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636587c7-2e2c-4ea7-bd84-adafa9a58d47 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:06.986154 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed44c0e1-c493-4f4d-8177-49a30f4d73e2 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:06.987747 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 944/1854] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:45:06 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:45:07.002845 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed44c0e1-c493-4f4d-8177-49a30f4d73e2 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:07.014908 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed26e4e-aa9c-4f4c-a536-4cc56a96e773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:07.016242 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed26e4e-aa9c-4f4c-a536-4cc56a96e773 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:07.016371 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed26e4e-aa9c-4f4c-a536-4cc56a96e773 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:07.016625 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed26e4e-aa9c-4f4c-a536-4cc56a96e773 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:07.017383 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed26e4e-aa9c-4f4c-a536-4cc56a96e773 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:07.019897 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed26e4e-aa9c-4f4c-a536-4cc56a96e773 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:07.053437 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 911/1855] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:45:06 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 17:45:07.057912 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636587c7-2e2c-4ea7-bd84-adafa9a58d47 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:07.072394 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636587c7-2e2c-4ea7-bd84-adafa9a58d47 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:07.079482 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3068d169-4e46-4fc6-9202-cbf6afc7907d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:07.080202 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3068d169-4e46-4fc6-9202-cbf6afc7907d admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:07.080484 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3068d169-4e46-4fc6-9202-cbf6afc7907d admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:07.080768 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3068d169-4e46-4fc6-9202-cbf6afc7907d admin admin] PATH_INFO: `/v3/projects/d333e36a18f74154a092b992d9ac79be/users/85a0dc279df249f9abf8a4f23df90c53/roles/11f8731756fe4caca13efa61b71b11c5` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:07.103275 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3068d169-4e46-4fc6-9202-cbf6afc7907d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d333e36a18f74154a092b992d9ac79be, user_id=85a0dc279df249f9abf8a4f23df90c53, role_id=11f8731756fe4caca13efa61b71b11c5)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:07.106712 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3068d169-4e46-4fc6-9202-cbf6afc7907d admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:07.135660 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 945/1856] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:45:07 2022] PUT /identity/v3/projects/d333e36a18f74154a092b992d9ac79be/users/85a0dc279df249f9abf8a4f23df90c53/roles/11f8731756fe4caca13efa61b71b11c5 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 17:45:07.139362 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0588e1e6-13e6-4776-ac32-8925bfc61127 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:07.140332 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0588e1e6-13e6-4776-ac32-8925bfc61127 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:07.141190 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0588e1e6-13e6-4776-ac32-8925bfc61127 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:07.198852 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3068d169-4e46-4fc6-9202-cbf6afc7907d admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:07.209634 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-3068d169-4e46-4fc6-9202-cbf6afc7907d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:07.227812 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-0588e1e6-13e6-4776-ac32-8925bfc61127 None None] MFA Rules not processed for user `85a0dc279df249f9abf8a4f23df90c53`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:45:07.229723 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-0588e1e6-13e6-4776-ac32-8925bfc61127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:07.342399 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:45:07.369590 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3068d169-4e46-4fc6-9202-cbf6afc7907d admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:07.376516 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceace704-4e91-489c-8481-9ad64807e201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:07.377464 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceace704-4e91-489c-8481-9ad64807e201 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:07.377824 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceace704-4e91-489c-8481-9ad64807e201 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:07.378064 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceace704-4e91-489c-8481-9ad64807e201 admin admin] PATH_INFO: `/v3/projects/2576b86cf8834856a824ae174e939069` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:07.379617 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceace704-4e91-489c-8481-9ad64807e201 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2576b86cf8834856a824ae174e939069)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:07.381867 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceace704-4e91-489c-8481-9ad64807e201 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:07.386330 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 912/1857] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:45:07 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:45:07.392810 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0588e1e6-13e6-4776-ac32-8925bfc61127 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:07.393309 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:45:07.409819 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0588e1e6-13e6-4776-ac32-8925bfc61127 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:07.419290 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bc3c5a-9e22-454d-a250-6fdb2a527c9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:07.420672 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bc3c5a-9e22-454d-a250-6fdb2a527c9e admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:07.421815 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bc3c5a-9e22-454d-a250-6fdb2a527c9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:07.422411 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bc3c5a-9e22-454d-a250-6fdb2a527c9e admin admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:07.423567 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bc3c5a-9e22-454d-a250-6fdb2a527c9e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:07.426056 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bc3c5a-9e22-454d-a250-6fdb2a527c9e admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:07.461585 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-ceace704-4e91-489c-8481-9ad64807e201 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2576b86cf8834856a824ae174e939069', 'request_id': 'req-ceace704-4e91-489c-8481-9ad64807e201'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:07.472373 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-ceace704-4e91-489c-8481-9ad64807e201 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2576b86cf8834856a824ae174e939069', 'request_id': 'req-ceace704-4e91-489c-8481-9ad64807e201'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:07.484120 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-ceace704-4e91-489c-8481-9ad64807e201 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2576b86cf8834856a824ae174e939069', 'request_id': 'req-ceace704-4e91-489c-8481-9ad64807e201'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:07.485116 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 913/1858] 10.222.0.89 () {60 vars in 1153 bytes} [Thu Jul 28 17:45:07 2022] POST /identity/v3/projects => generated 365 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:45:07.486280 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bc3c5a-9e22-454d-a250-6fdb2a527c9e admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:07.508137 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-ceace704-4e91-489c-8481-9ad64807e201 admin admin] The token cache is being invalidate because project 2576b86cf8834856a824ae174e939069 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80324) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 17:45:07.508602 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-ceace704-4e91-489c-8481-9ad64807e201 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:07.509859 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-ceace704-4e91-489c-8481-9ad64807e201 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:07.511964 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 946/1859] 10.222.0.89 () {58 vars in 1233 bytes} [Thu Jul 28 17:45:07 2022] DELETE /identity/v3/projects/2576b86cf8834856a824ae174e939069 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:45:07.517088 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceace704-4e91-489c-8481-9ad64807e201 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:07.518838 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-ceace704-4e91-489c-8481-9ad64807e201 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:07.630902 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceace704-4e91-489c-8481-9ad64807e201 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:07.648517 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bc3c5a-9e22-454d-a250-6fdb2a527c9e admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:07.653468 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2010c509-4335-4c66-91a5-d5bd438a7846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:07.654398 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2010c509-4335-4c66-91a5-d5bd438a7846 admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:07.654765 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2010c509-4335-4c66-91a5-d5bd438a7846 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:07.655282 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2010c509-4335-4c66-91a5-d5bd438a7846 admin admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:07.656705 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2010c509-4335-4c66-91a5-d5bd438a7846 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:07.658885 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2010c509-4335-4c66-91a5-d5bd438a7846 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:07.690617 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 914/1860] 10.222.0.89 () {60 vars in 1144 bytes} [Thu Jul 28 17:45:07 2022] POST /identity/v3/users => generated 289 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:45:07.691657 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2010c509-4335-4c66-91a5-d5bd438a7846 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:07.693580 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-2010c509-4335-4c66-91a5-d5bd438a7846 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:07.708768 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2010c509-4335-4c66-91a5-d5bd438a7846 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:07.714454 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc45bd30-f353-4d81-b89a-8a31b586c15f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:07.715151 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc45bd30-f353-4d81-b89a-8a31b586c15f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:07.715444 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d45316-8e9d-45ea-b9d9-96fd80d32982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:07.715759 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc45bd30-f353-4d81-b89a-8a31b586c15f admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:07.716174 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc45bd30-f353-4d81-b89a-8a31b586c15f admin admin] PATH_INFO: `/v3/users/32a305c1dfa2485588b272d9c7d53bcc` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:07.716354 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d45316-8e9d-45ea-b9d9-96fd80d32982 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:07.716707 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d45316-8e9d-45ea-b9d9-96fd80d32982 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:07.716967 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d45316-8e9d-45ea-b9d9-96fd80d32982 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:07.717501 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc45bd30-f353-4d81-b89a-8a31b586c15f admin admin] RBAC: Authorizing `identity:delete_user(user_id=32a305c1dfa2485588b272d9c7d53bcc)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:07.718212 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d45316-8e9d-45ea-b9d9-96fd80d32982 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:07.720171 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc45bd30-f353-4d81-b89a-8a31b586c15f admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:07.720274 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d45316-8e9d-45ea-b9d9-96fd80d32982 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:07.731934 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 915/1861] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:45:07 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:45:07.738018 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d45316-8e9d-45ea-b9d9-96fd80d32982 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:07.748980 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d45316-8e9d-45ea-b9d9-96fd80d32982 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:07.755728 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11da896c-58af-459e-a5a3-3390ad54cfed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:07.756473 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11da896c-58af-459e-a5a3-3390ad54cfed admin admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:07.756755 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11da896c-58af-459e-a5a3-3390ad54cfed admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:07.757054 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11da896c-58af-459e-a5a3-3390ad54cfed admin admin] PATH_INFO: `/v3/projects/7463ef96c10a45d1903b7c38e55bdb04/users/abc1100721eb4316b7265bcdcb14b7a5/roles/73e457231a824ea98c28d22610be640c` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:07.775792 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11da896c-58af-459e-a5a3-3390ad54cfed admin admin] RBAC: Authorizing `identity:create_grant(project_id=7463ef96c10a45d1903b7c38e55bdb04, user_id=abc1100721eb4316b7265bcdcb14b7a5, role_id=73e457231a824ea98c28d22610be640c)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:07.779474 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11da896c-58af-459e-a5a3-3390ad54cfed admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:07.782953 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:45:07.785134 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-dc45bd30-f353-4d81-b89a-8a31b586c15f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32a305c1dfa2485588b272d9c7d53bcc'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:07.797872 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-dc45bd30-f353-4d81-b89a-8a31b586c15f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32a305c1dfa2485588b272d9c7d53bcc'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:07.798981 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-dc45bd30-f353-4d81-b89a-8a31b586c15f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32a305c1dfa2485588b272d9c7d53bcc'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:07.804107 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 916/1862] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:45:07 2022] PUT /identity/v3/projects/7463ef96c10a45d1903b7c38e55bdb04/users/abc1100721eb4316b7265bcdcb14b7a5/roles/73e457231a824ea98c28d22610be640c => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 17:45:07.805213 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11da896c-58af-459e-a5a3-3390ad54cfed admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:07.807684 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-11da896c-58af-459e-a5a3-3390ad54cfed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:07.814428 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-dc45bd30-f353-4d81-b89a-8a31b586c15f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32a305c1dfa2485588b272d9c7d53bcc'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:07.833376 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-dc45bd30-f353-4d81-b89a-8a31b586c15f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32a305c1dfa2485588b272d9c7d53bcc'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:07.841382 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 947/1863] 10.222.0.89 () {58 vars in 1224 bytes} [Thu Jul 28 17:45:07 2022] DELETE /identity/v3/users/32a305c1dfa2485588b272d9c7d53bcc => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:45:07.897137 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11da896c-58af-459e-a5a3-3390ad54cfed admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:07.923881 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:45:07.999083 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9dcde5-1e45-4ea8-aeaa-154f993ae8b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:07.999868 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9dcde5-1e45-4ea8-aeaa-154f993ae8b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:08.000179 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9dcde5-1e45-4ea8-aeaa-154f993ae8b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:08.000493 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9dcde5-1e45-4ea8-aeaa-154f993ae8b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:08.001255 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9dcde5-1e45-4ea8-aeaa-154f993ae8b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:08.003286 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9dcde5-1e45-4ea8-aeaa-154f993ae8b3 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:08.018016 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 917/1864] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:45:07 2022] GET /identity/v3/roles => generated 1420 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:45:08.022293 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc45bd30-f353-4d81-b89a-8a31b586c15f admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:08.034173 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc45bd30-f353-4d81-b89a-8a31b586c15f admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:08.040090 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3fa192-decd-482f-b604-0ce6a511f7e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:08.040845 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3fa192-decd-482f-b604-0ce6a511f7e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:08.041112 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3fa192-decd-482f-b604-0ce6a511f7e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:08.041679 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3fa192-decd-482f-b604-0ce6a511f7e9 admin admin] PATH_INFO: `/v3/projects/7463ef96c10a45d1903b7c38e55bdb04/users/abc1100721eb4316b7265bcdcb14b7a5/roles/73e457231a824ea98c28d22610be640c` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:08.043997 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3fa192-decd-482f-b604-0ce6a511f7e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7463ef96c10a45d1903b7c38e55bdb04, user_id=abc1100721eb4316b7265bcdcb14b7a5, role_id=73e457231a824ea98c28d22610be640c)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:08.047073 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3fa192-decd-482f-b604-0ce6a511f7e9 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:08.080870 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 948/1865] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:45:08 2022] PUT /identity/v3/projects/7463ef96c10a45d1903b7c38e55bdb04/users/abc1100721eb4316b7265bcdcb14b7a5/roles/73e457231a824ea98c28d22610be640c => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 17:45:08.092145 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6c8ba6-4d34-4263-beff-8cf6bb3a526e None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:08.092559 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6c8ba6-4d34-4263-beff-8cf6bb3a526e None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:08.092683 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6c8ba6-4d34-4263-beff-8cf6bb3a526e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:08.130259 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-fa6c8ba6-4d34-4263-beff-8cf6bb3a526e None None] MFA Rules not processed for user `abc1100721eb4316b7265bcdcb14b7a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:45:08.131797 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-fa6c8ba6-4d34-4263-beff-8cf6bb3a526e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:08.223179 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 918/1866] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:45:08 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 17:45:08.236675 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3fa192-decd-482f-b604-0ce6a511f7e9 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:08.244935 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-8b3fa192-decd-482f-b604-0ce6a511f7e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:08.334897 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3fa192-decd-482f-b604-0ce6a511f7e9 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:08.338580 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a296b4f-491f-4cc1-8d27-7308dc84ef05 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96d512d0c84240918dae6c4a3e8b7b79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:08.339192 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a296b4f-491f-4cc1-8d27-7308dc84ef05 service cinder] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:08.339468 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a296b4f-491f-4cc1-8d27-7308dc84ef05 service cinder] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:08.339738 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a296b4f-491f-4cc1-8d27-7308dc84ef05 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:08.350880 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a296b4f-491f-4cc1-8d27-7308dc84ef05 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:08.352596 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a296b4f-491f-4cc1-8d27-7308dc84ef05 service cinder] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:08.381012 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 949/1867] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:45:08 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:45:08.467075 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6c8ba6-4d34-4263-beff-8cf6bb3a526e None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:08.477638 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6c8ba6-4d34-4263-beff-8cf6bb3a526e None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:08.482741 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6193a49-1d75-4f8f-8b8d-092cadd5a3d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96d512d0c84240918dae6c4a3e8b7b79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:08.484101 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6193a49-1d75-4f8f-8b8d-092cadd5a3d0 service cinder] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:08.484500 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6193a49-1d75-4f8f-8b8d-092cadd5a3d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:08.484888 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6193a49-1d75-4f8f-8b8d-092cadd5a3d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:08.487814 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-e6193a49-1d75-4f8f-8b8d-092cadd5a3d0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:08.575117 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6193a49-1d75-4f8f-8b8d-092cadd5a3d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:08.576870 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6193a49-1d75-4f8f-8b8d-092cadd5a3d0 service cinder] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:08.588030 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 919/1868] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:45:08 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:45:08.928611 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a296b4f-491f-4cc1-8d27-7308dc84ef05 service cinder] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:08.930098 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-1a296b4f-491f-4cc1-8d27-7308dc84ef05 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:09.032211 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a296b4f-491f-4cc1-8d27-7308dc84ef05 service cinder] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:09.037238 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2e2b4c-f7e8-4d6d-a4ab-a7d84795b333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:09.037887 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2e2b4c-f7e8-4d6d-a4ab-a7d84795b333 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:09.038172 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2e2b4c-f7e8-4d6d-a4ab-a7d84795b333 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:09.038437 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2e2b4c-f7e8-4d6d-a4ab-a7d84795b333 admin admin] PATH_INFO: `/v3/projects/c7fd42cc001142279634135b4cb6727c` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:09.039640 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2e2b4c-f7e8-4d6d-a4ab-a7d84795b333 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7fd42cc001142279634135b4cb6727c)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:09.041579 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2e2b4c-f7e8-4d6d-a4ab-a7d84795b333 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:09.099081 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-8c2e2b4c-f7e8-4d6d-a4ab-a7d84795b333 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7fd42cc001142279634135b4cb6727c', 'request_id': 'req-8c2e2b4c-f7e8-4d6d-a4ab-a7d84795b333'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:09.103303 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:45:09.105961 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-8c2e2b4c-f7e8-4d6d-a4ab-a7d84795b333 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7fd42cc001142279634135b4cb6727c', 'request_id': 'req-8c2e2b4c-f7e8-4d6d-a4ab-a7d84795b333'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:09.117668 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-8c2e2b4c-f7e8-4d6d-a4ab-a7d84795b333 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7fd42cc001142279634135b4cb6727c', 'request_id': 'req-8c2e2b4c-f7e8-4d6d-a4ab-a7d84795b333'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:09.135165 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-8c2e2b4c-f7e8-4d6d-a4ab-a7d84795b333 admin admin] The token cache is being invalidate because project c7fd42cc001142279634135b4cb6727c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80324) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 17:45:09.135459 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-8c2e2b4c-f7e8-4d6d-a4ab-a7d84795b333 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:09.137071 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-8c2e2b4c-f7e8-4d6d-a4ab-a7d84795b333 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:09.139242 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 950/1869] 10.222.0.89 () {58 vars in 1233 bytes} [Thu Jul 28 17:45:08 2022] DELETE /identity/v3/projects/c7fd42cc001142279634135b4cb6727c => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:45:09.144382 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4163c1eb-d4e5-45a2-ad18-b40fcbe2d1ab None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:09.144836 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4163c1eb-d4e5-45a2-ad18-b40fcbe2d1ab None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:09.144998 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4163c1eb-d4e5-45a2-ad18-b40fcbe2d1ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:09.172610 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-4163c1eb-d4e5-45a2-ad18-b40fcbe2d1ab None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:45:09.173948 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-4163c1eb-d4e5-45a2-ad18-b40fcbe2d1ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:09.216197 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:45:09.264847 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 920/1870] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:45:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 17:45:09.266962 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54daa0c3-8193-499a-beb2-035355c09a11 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:09.267511 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54daa0c3-8193-499a-beb2-035355c09a11 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:09.267617 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54daa0c3-8193-499a-beb2-035355c09a11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:09.290087 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-54daa0c3-8193-499a-beb2-035355c09a11 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:45:09.294169 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-54daa0c3-8193-499a-beb2-035355c09a11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:09.305631 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 951/1871] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:45:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:45:09.309898 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4163c1eb-d4e5-45a2-ad18-b40fcbe2d1ab None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:09.321950 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4163c1eb-d4e5-45a2-ad18-b40fcbe2d1ab None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:09.326745 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82fc238-973d-4d8b-86f9-4258b9f746f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:09.328290 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82fc238-973d-4d8b-86f9-4258b9f746f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:09.328704 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82fc238-973d-4d8b-86f9-4258b9f746f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:09.328978 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82fc238-973d-4d8b-86f9-4258b9f746f3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:09.330179 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82fc238-973d-4d8b-86f9-4258b9f746f3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:45:09.330486 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82fc238-973d-4d8b-86f9-4258b9f746f3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:09.332093 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82fc238-973d-4d8b-86f9-4258b9f746f3 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:09.350681 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 921/1872] 10.222.0.89 () {58 vars in 1153 bytes} [Thu Jul 28 17:45:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:45:09.351786 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54daa0c3-8193-499a-beb2-035355c09a11 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:09.365176 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54daa0c3-8193-499a-beb2-035355c09a11 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:09.373328 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d2c858-ddd1-49cd-8e26-ea53d17be1f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:09.374108 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d2c858-ddd1-49cd-8e26-ea53d17be1f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:09.374666 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d2c858-ddd1-49cd-8e26-ea53d17be1f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:09.375411 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d2c858-ddd1-49cd-8e26-ea53d17be1f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:09.376095 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d2c858-ddd1-49cd-8e26-ea53d17be1f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:09.378410 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d2c858-ddd1-49cd-8e26-ea53d17be1f6 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:09.423518 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82fc238-973d-4d8b-86f9-4258b9f746f3 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:09.425077 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 952/1873] 10.222.0.89 () {60 vars in 1153 bytes} [Thu Jul 28 17:45:09 2022] POST /identity/v3/projects => generated 359 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:45:09.448421 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82fc238-973d-4d8b-86f9-4258b9f746f3 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:09.559194 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69e5350-f68e-44f6-8ccb-2228d9e43fad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:09.559833 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69e5350-f68e-44f6-8ccb-2228d9e43fad admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:09.560117 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69e5350-f68e-44f6-8ccb-2228d9e43fad admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:09.560394 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69e5350-f68e-44f6-8ccb-2228d9e43fad admin admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:09.561083 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69e5350-f68e-44f6-8ccb-2228d9e43fad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:09.563182 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69e5350-f68e-44f6-8ccb-2228d9e43fad admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:09.592255 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 922/1874] 10.222.0.89 () {60 vars in 1144 bytes} [Thu Jul 28 17:45:09 2022] POST /identity/v3/users => generated 280 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 17:45:09.595936 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d2c858-ddd1-49cd-8e26-ea53d17be1f6 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:09.617441 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d2c858-ddd1-49cd-8e26-ea53d17be1f6 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:09.622191 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94acb35-5663-4859-9923-f1d8f5407f61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:09.622846 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94acb35-5663-4859-9923-f1d8f5407f61 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:09.623085 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94acb35-5663-4859-9923-f1d8f5407f61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:09.623360 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94acb35-5663-4859-9923-f1d8f5407f61 admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:09.624088 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94acb35-5663-4859-9923-f1d8f5407f61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:09.626255 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94acb35-5663-4859-9923-f1d8f5407f61 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:09.645199 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 953/1875] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:45:09 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:45:09.646450 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69e5350-f68e-44f6-8ccb-2228d9e43fad admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:09.659543 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69e5350-f68e-44f6-8ccb-2228d9e43fad admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:09.664610 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8922670-faa6-4263-aa34-bdebabae6bb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:09.665244 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8922670-faa6-4263-aa34-bdebabae6bb9 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:09.665527 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8922670-faa6-4263-aa34-bdebabae6bb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:09.665795 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8922670-faa6-4263-aa34-bdebabae6bb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:09.666509 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8922670-faa6-4263-aa34-bdebabae6bb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:09.668159 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8922670-faa6-4263-aa34-bdebabae6bb9 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:09.684338 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 923/1876] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:45:09 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:45:09.685777 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94acb35-5663-4859-9923-f1d8f5407f61 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:09.697010 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94acb35-5663-4859-9923-f1d8f5407f61 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:09.701948 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e293376-a394-4bfd-aae0-dddf84324e69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:09.703026 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e293376-a394-4bfd-aae0-dddf84324e69 admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:09.703335 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e293376-a394-4bfd-aae0-dddf84324e69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:09.703591 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e293376-a394-4bfd-aae0-dddf84324e69 admin admin] PATH_INFO: `/v3/projects/2a75ab1707a9432c90599817bf2a9bd1/users/914a810b856f4434b6d1ef425d4934b4/roles/11f8731756fe4caca13efa61b71b11c5` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:09.721350 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e293376-a394-4bfd-aae0-dddf84324e69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a75ab1707a9432c90599817bf2a9bd1, user_id=914a810b856f4434b6d1ef425d4934b4, role_id=11f8731756fe4caca13efa61b71b11c5)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:09.724314 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e293376-a394-4bfd-aae0-dddf84324e69 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:09.741514 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 954/1877] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:45:09 2022] PUT /identity/v3/projects/2a75ab1707a9432c90599817bf2a9bd1/users/914a810b856f4434b6d1ef425d4934b4/roles/11f8731756fe4caca13efa61b71b11c5 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:45:09.753916 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aa2601-118e-4d4c-a658-c701eff5aef8 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:09.754202 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aa2601-118e-4d4c-a658-c701eff5aef8 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:09.754552 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aa2601-118e-4d4c-a658-c701eff5aef8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:09.788428 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-93aa2601-118e-4d4c-a658-c701eff5aef8 None None] MFA Rules not processed for user `914a810b856f4434b6d1ef425d4934b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:45:09.789795 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-93aa2601-118e-4d4c-a658-c701eff5aef8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:09.982754 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 924/1878] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:45:09 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:45:10.037807 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e293376-a394-4bfd-aae0-dddf84324e69 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:10.039629 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-5e293376-a394-4bfd-aae0-dddf84324e69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:10.161333 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e293376-a394-4bfd-aae0-dddf84324e69 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:10.166761 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc6508d-7845-489b-8457-3d03944959c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96d512d0c84240918dae6c4a3e8b7b79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:10.167400 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc6508d-7845-489b-8457-3d03944959c2 service cinder] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:10.167659 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc6508d-7845-489b-8457-3d03944959c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:10.167938 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc6508d-7845-489b-8457-3d03944959c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:10.179266 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc6508d-7845-489b-8457-3d03944959c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:10.181552 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc6508d-7845-489b-8457-3d03944959c2 service cinder] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:10.195600 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 955/1879] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:45:10 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:45:10.216694 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93aa2601-118e-4d4c-a658-c701eff5aef8 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:10.298762 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:45:10.392896 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93aa2601-118e-4d4c-a658-c701eff5aef8 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:10.397890 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5bdf6e-184f-4a21-9030-decaf2c135db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:10.398613 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5bdf6e-184f-4a21-9030-decaf2c135db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:10.399056 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5bdf6e-184f-4a21-9030-decaf2c135db admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:10.399299 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5bdf6e-184f-4a21-9030-decaf2c135db admin admin] PATH_INFO: `/v3/users/914a810b856f4434b6d1ef425d4934b4` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:10.400707 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5bdf6e-184f-4a21-9030-decaf2c135db admin admin] RBAC: Authorizing `identity:delete_user(user_id=914a810b856f4434b6d1ef425d4934b4)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:10.402524 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5bdf6e-184f-4a21-9030-decaf2c135db admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:10.473404 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-4f5bdf6e-184f-4a21-9030-decaf2c135db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '914a810b856f4434b6d1ef425d4934b4'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:10.503440 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-4f5bdf6e-184f-4a21-9030-decaf2c135db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '914a810b856f4434b6d1ef425d4934b4'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:10.531832 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-4f5bdf6e-184f-4a21-9030-decaf2c135db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '914a810b856f4434b6d1ef425d4934b4'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:10.532605 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-4f5bdf6e-184f-4a21-9030-decaf2c135db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '914a810b856f4434b6d1ef425d4934b4'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:10.534678 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-4f5bdf6e-184f-4a21-9030-decaf2c135db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '914a810b856f4434b6d1ef425d4934b4'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:10.564892 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 925/1880] 10.222.0.89 () {58 vars in 1224 bytes} [Thu Jul 28 17:45:10 2022] DELETE /identity/v3/users/914a810b856f4434b6d1ef425d4934b4 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:45:10.567912 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90cbab3-5ea3-4b7e-9dd4-8d11194662ae None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:10.568388 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90cbab3-5ea3-4b7e-9dd4-8d11194662ae None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:10.568846 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90cbab3-5ea3-4b7e-9dd4-8d11194662ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:10.606646 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-c90cbab3-5ea3-4b7e-9dd4-8d11194662ae None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:45:10.606878 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-c90cbab3-5ea3-4b7e-9dd4-8d11194662ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:10.738233 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 956/1881] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:45:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:45:10.777013 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5bdf6e-184f-4a21-9030-decaf2c135db admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:10.784003 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-4f5bdf6e-184f-4a21-9030-decaf2c135db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:10.820078 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90cbab3-5ea3-4b7e-9dd4-8d11194662ae None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:10.821917 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-c90cbab3-5ea3-4b7e-9dd4-8d11194662ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:10.850272 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90cbab3-5ea3-4b7e-9dd4-8d11194662ae None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:10.864460 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4b49f5-efcc-4400-9c1d-a16289c03133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:10.865330 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4b49f5-efcc-4400-9c1d-a16289c03133 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:10.865723 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4b49f5-efcc-4400-9c1d-a16289c03133 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:10.866104 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4b49f5-efcc-4400-9c1d-a16289c03133 admin admin] PATH_INFO: `/v3/users/31f8a00531a341c99161aef8f8ea9c9c` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:10.867867 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4b49f5-efcc-4400-9c1d-a16289c03133 admin admin] RBAC: Authorizing `identity:delete_user(user_id=31f8a00531a341c99161aef8f8ea9c9c)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:10.870667 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4b49f5-efcc-4400-9c1d-a16289c03133 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:10.894165 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5bdf6e-184f-4a21-9030-decaf2c135db admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:10.900763 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aea9f6b-df40-4253-abcc-22af86f48373 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76817cd541294331b0023069092e8e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:10.901621 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aea9f6b-df40-4253-abcc-22af86f48373 service neutron] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:10.902004 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aea9f6b-df40-4253-abcc-22af86f48373 service neutron] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:10.902378 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aea9f6b-df40-4253-abcc-22af86f48373 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:10.909445 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:45:10.916736 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aea9f6b-df40-4253-abcc-22af86f48373 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:10.922642 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aea9f6b-df40-4253-abcc-22af86f48373 service neutron] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:10.939066 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 926/1882] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:45:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:45:10.974827 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-9c4b49f5-efcc-4400-9c1d-a16289c03133 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31f8a00531a341c99161aef8f8ea9c9c'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:10.992769 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-9c4b49f5-efcc-4400-9c1d-a16289c03133 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31f8a00531a341c99161aef8f8ea9c9c'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:10.993650 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-9c4b49f5-efcc-4400-9c1d-a16289c03133 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31f8a00531a341c99161aef8f8ea9c9c'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:11.003181 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-9c4b49f5-efcc-4400-9c1d-a16289c03133 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31f8a00531a341c99161aef8f8ea9c9c'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:11.018606 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-9c4b49f5-efcc-4400-9c1d-a16289c03133 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31f8a00531a341c99161aef8f8ea9c9c'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:11.029238 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 957/1883] 10.222.0.89 () {58 vars in 1224 bytes} [Thu Jul 28 17:45:10 2022] DELETE /identity/v3/users/31f8a00531a341c99161aef8f8ea9c9c => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 17:45:11.031058 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8503d4ed-a0da-4754-9947-c139d60b4a5a None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:11.031350 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8503d4ed-a0da-4754-9947-c139d60b4a5a None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:11.031827 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8503d4ed-a0da-4754-9947-c139d60b4a5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:11.061019 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-8503d4ed-a0da-4754-9947-c139d60b4a5a None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:45:11.062659 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-8503d4ed-a0da-4754-9947-c139d60b4a5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:11.126531 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:45:11.145790 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 927/1884] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:45:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:45:11.187176 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4b49f5-efcc-4400-9c1d-a16289c03133 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:11.189209 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-9c4b49f5-efcc-4400-9c1d-a16289c03133 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:11.279505 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4b49f5-efcc-4400-9c1d-a16289c03133 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:11.287221 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0473fc18-c344-4ce4-928f-1cdc293d6238 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76817cd541294331b0023069092e8e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:11.287974 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0473fc18-c344-4ce4-928f-1cdc293d6238 service neutron] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:11.289613 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0473fc18-c344-4ce4-928f-1cdc293d6238 service neutron] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:11.289954 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0473fc18-c344-4ce4-928f-1cdc293d6238 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:11.307956 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0473fc18-c344-4ce4-928f-1cdc293d6238 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:11.310250 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0473fc18-c344-4ce4-928f-1cdc293d6238 service neutron] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:11.329079 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 958/1885] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:45:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:45:11.887358 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8503d4ed-a0da-4754-9947-c139d60b4a5a None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:11.888847 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-8503d4ed-a0da-4754-9947-c139d60b4a5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:11.914504 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8503d4ed-a0da-4754-9947-c139d60b4a5a None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:11.941067 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26367c82-6b2e-4d79-ab32-a9d58a31374b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:11.941067 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26367c82-6b2e-4d79-ab32-a9d58a31374b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:11.941067 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26367c82-6b2e-4d79-ab32-a9d58a31374b admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:11.941067 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26367c82-6b2e-4d79-ab32-a9d58a31374b admin admin] PATH_INFO: `/v3/projects/2a75ab1707a9432c90599817bf2a9bd1` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:11.949954 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26367c82-6b2e-4d79-ab32-a9d58a31374b admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a75ab1707a9432c90599817bf2a9bd1)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:11.957410 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26367c82-6b2e-4d79-ab32-a9d58a31374b admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:11.995451 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0473fc18-c344-4ce4-928f-1cdc293d6238 service neutron] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:12.002117 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-0473fc18-c344-4ce4-928f-1cdc293d6238 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:12.021044 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0473fc18-c344-4ce4-928f-1cdc293d6238 service neutron] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:12.043306 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-26367c82-6b2e-4d79-ab32-a9d58a31374b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a75ab1707a9432c90599817bf2a9bd1', 'request_id': 'req-26367c82-6b2e-4d79-ab32-a9d58a31374b'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:12.053670 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-26367c82-6b2e-4d79-ab32-a9d58a31374b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a75ab1707a9432c90599817bf2a9bd1', 'request_id': 'req-26367c82-6b2e-4d79-ab32-a9d58a31374b'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:12.063044 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-26367c82-6b2e-4d79-ab32-a9d58a31374b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a75ab1707a9432c90599817bf2a9bd1', 'request_id': 'req-26367c82-6b2e-4d79-ab32-a9d58a31374b'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:12.093579 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-26367c82-6b2e-4d79-ab32-a9d58a31374b admin admin] The token cache is being invalidate because project 2a75ab1707a9432c90599817bf2a9bd1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80325) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 17:45:12.093875 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-26367c82-6b2e-4d79-ab32-a9d58a31374b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:12.094850 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-26367c82-6b2e-4d79-ab32-a9d58a31374b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:12.097480 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 928/1886] 10.222.0.89 () {58 vars in 1233 bytes} [Thu Jul 28 17:45:11 2022] DELETE /identity/v3/projects/2a75ab1707a9432c90599817bf2a9bd1 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:45:12.098451 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26367c82-6b2e-4d79-ab32-a9d58a31374b admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:12.100776 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-26367c82-6b2e-4d79-ab32-a9d58a31374b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:12.122871 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b69c10-c08f-41a4-84ab-118bebf74fba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:12.123525 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b69c10-c08f-41a4-84ab-118bebf74fba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:12.124521 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b69c10-c08f-41a4-84ab-118bebf74fba admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:12.125099 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b69c10-c08f-41a4-84ab-118bebf74fba admin admin] PATH_INFO: `/v3/users/85a0dc279df249f9abf8a4f23df90c53` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:12.126749 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b69c10-c08f-41a4-84ab-118bebf74fba admin admin] RBAC: Authorizing `identity:delete_user(user_id=85a0dc279df249f9abf8a4f23df90c53)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:12.128925 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b69c10-c08f-41a4-84ab-118bebf74fba admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:12.190479 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:45:12.199011 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-68b69c10-c08f-41a4-84ab-118bebf74fba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85a0dc279df249f9abf8a4f23df90c53'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:12.212326 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-68b69c10-c08f-41a4-84ab-118bebf74fba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85a0dc279df249f9abf8a4f23df90c53'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:12.215303 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-68b69c10-c08f-41a4-84ab-118bebf74fba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85a0dc279df249f9abf8a4f23df90c53'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:12.221778 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26367c82-6b2e-4d79-ab32-a9d58a31374b admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:12.230214 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb75bc9f-8ae7-49e2-8c77-725c18a472e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:12.232588 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb75bc9f-8ae7-49e2-8c77-725c18a472e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:12.233422 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-68b69c10-c08f-41a4-84ab-118bebf74fba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85a0dc279df249f9abf8a4f23df90c53'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:12.234466 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb75bc9f-8ae7-49e2-8c77-725c18a472e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:12.234973 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb75bc9f-8ae7-49e2-8c77-725c18a472e8 admin admin] PATH_INFO: `/v3/projects/d995f269855747cfa22b6ddb8f66db71` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:12.237470 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb75bc9f-8ae7-49e2-8c77-725c18a472e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d995f269855747cfa22b6ddb8f66db71)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:12.240371 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb75bc9f-8ae7-49e2-8c77-725c18a472e8 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:12.256224 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-68b69c10-c08f-41a4-84ab-118bebf74fba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85a0dc279df249f9abf8a4f23df90c53'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:12.272514 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 959/1887] 10.222.0.89 () {58 vars in 1224 bytes} [Thu Jul 28 17:45:11 2022] DELETE /identity/v3/users/85a0dc279df249f9abf8a4f23df90c53 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 17:45:12.273953 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b30e8b-70f6-462c-8a83-fd0397a8cee6 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:12.274240 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b30e8b-70f6-462c-8a83-fd0397a8cee6 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:12.274665 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b30e8b-70f6-462c-8a83-fd0397a8cee6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:12.310758 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-b8b30e8b-70f6-462c-8a83-fd0397a8cee6 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:45:12.312885 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-b8b30e8b-70f6-462c-8a83-fd0397a8cee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:12.340887 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-fb75bc9f-8ae7-49e2-8c77-725c18a472e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd995f269855747cfa22b6ddb8f66db71', 'request_id': 'req-fb75bc9f-8ae7-49e2-8c77-725c18a472e8'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:12.349786 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-fb75bc9f-8ae7-49e2-8c77-725c18a472e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd995f269855747cfa22b6ddb8f66db71', 'request_id': 'req-fb75bc9f-8ae7-49e2-8c77-725c18a472e8'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:12.359806 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-fb75bc9f-8ae7-49e2-8c77-725c18a472e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd995f269855747cfa22b6ddb8f66db71', 'request_id': 'req-fb75bc9f-8ae7-49e2-8c77-725c18a472e8'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:12.405457 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-fb75bc9f-8ae7-49e2-8c77-725c18a472e8 admin admin] The token cache is being invalidate because project d995f269855747cfa22b6ddb8f66db71 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80325) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 17:45:12.407856 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-fb75bc9f-8ae7-49e2-8c77-725c18a472e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:12.409031 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-fb75bc9f-8ae7-49e2-8c77-725c18a472e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:12.415096 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 929/1888] 10.222.0.89 () {58 vars in 1233 bytes} [Thu Jul 28 17:45:12 2022] DELETE /identity/v3/projects/d995f269855747cfa22b6ddb8f66db71 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:45:12.416938 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32d07e8-9023-431c-8781-02b31acff3e9 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:12.417421 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32d07e8-9023-431c-8781-02b31acff3e9 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:12.417836 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32d07e8-9023-431c-8781-02b31acff3e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:12.532774 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-e32d07e8-9023-431c-8781-02b31acff3e9 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:45:12.535813 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-e32d07e8-9023-431c-8781-02b31acff3e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:12.562560 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 960/1889] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:45:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:45:12.566404 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b30e8b-70f6-462c-8a83-fd0397a8cee6 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:12.574773 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-b8b30e8b-70f6-462c-8a83-fd0397a8cee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:12.600840 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:45:12.615679 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b30e8b-70f6-462c-8a83-fd0397a8cee6 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:12.713307 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 930/1890] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:45:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:45:12.716815 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded97c81-f873-4e03-9bae-b6a2d223c1a5 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:12.718167 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded97c81-f873-4e03-9bae-b6a2d223c1a5 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:12.720031 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded97c81-f873-4e03-9bae-b6a2d223c1a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:12.724981 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8a5fbf-00a9-4439-b6f2-2f7528fc2f40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:12.727849 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8a5fbf-00a9-4439-b6f2-2f7528fc2f40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:12.727849 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8a5fbf-00a9-4439-b6f2-2f7528fc2f40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:12.727849 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8a5fbf-00a9-4439-b6f2-2f7528fc2f40 admin admin] PATH_INFO: `/v3/users/db543e5ad1064ffab11099ca204ea86f` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:12.728262 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8a5fbf-00a9-4439-b6f2-2f7528fc2f40 admin admin] RBAC: Authorizing `identity:delete_user(user_id=db543e5ad1064ffab11099ca204ea86f)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:12.731297 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8a5fbf-00a9-4439-b6f2-2f7528fc2f40 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:12.748278 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-ded97c81-f873-4e03-9bae-b6a2d223c1a5 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:45:12.750463 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-ded97c81-f873-4e03-9bae-b6a2d223c1a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:12.760221 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 931/1891] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:45:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:45:12.761369 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded97c81-f873-4e03-9bae-b6a2d223c1a5 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:12.763000 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-ded97c81-f873-4e03-9bae-b6a2d223c1a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:12.812487 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-da8a5fbf-00a9-4439-b6f2-2f7528fc2f40 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db543e5ad1064ffab11099ca204ea86f'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:12.827126 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-da8a5fbf-00a9-4439-b6f2-2f7528fc2f40 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db543e5ad1064ffab11099ca204ea86f'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:12.829412 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-da8a5fbf-00a9-4439-b6f2-2f7528fc2f40 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db543e5ad1064ffab11099ca204ea86f'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:12.833628 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:45:12.858581 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-da8a5fbf-00a9-4439-b6f2-2f7528fc2f40 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db543e5ad1064ffab11099ca204ea86f'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:12.873430 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-da8a5fbf-00a9-4439-b6f2-2f7528fc2f40 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db543e5ad1064ffab11099ca204ea86f'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:12.881941 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 961/1892] 10.222.0.89 () {58 vars in 1224 bytes} [Thu Jul 28 17:45:12 2022] DELETE /identity/v3/users/db543e5ad1064ffab11099ca204ea86f => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:45:12.882980 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8a5fbf-00a9-4439-b6f2-2f7528fc2f40 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:12.885591 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-da8a5fbf-00a9-4439-b6f2-2f7528fc2f40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:12.901627 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded97c81-f873-4e03-9bae-b6a2d223c1a5 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:12.971796 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8a5fbf-00a9-4439-b6f2-2f7528fc2f40 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:12.978584 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea328bd8-c9e5-4e6a-a3a0-96329eff2e91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:12.979228 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea328bd8-c9e5-4e6a-a3a0-96329eff2e91 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:12.979688 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea328bd8-c9e5-4e6a-a3a0-96329eff2e91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:12.979973 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea328bd8-c9e5-4e6a-a3a0-96329eff2e91 admin admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:12.980687 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea328bd8-c9e5-4e6a-a3a0-96329eff2e91 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:45:12.980999 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea328bd8-c9e5-4e6a-a3a0-96329eff2e91 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:12.982963 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea328bd8-c9e5-4e6a-a3a0-96329eff2e91 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:13.000839 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 962/1893] 10.222.0.89 () {58 vars in 1153 bytes} [Thu Jul 28 17:45:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:45:13.006844 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea328bd8-c9e5-4e6a-a3a0-96329eff2e91 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:13.019822 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea328bd8-c9e5-4e6a-a3a0-96329eff2e91 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:13.025412 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c92ed7b-f3f9-46bf-89dd-265e308fae44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:13.026724 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c92ed7b-f3f9-46bf-89dd-265e308fae44 admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:13.027011 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c92ed7b-f3f9-46bf-89dd-265e308fae44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:13.027290 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c92ed7b-f3f9-46bf-89dd-265e308fae44 admin admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:13.027969 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c92ed7b-f3f9-46bf-89dd-265e308fae44 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:13.029979 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c92ed7b-f3f9-46bf-89dd-265e308fae44 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:13.031346 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ee7541-c6e6-420f-a791-503cb0279663 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76817cd541294331b0023069092e8e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:13.035357 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ee7541-c6e6-420f-a791-503cb0279663 service neutron] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:13.035357 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ee7541-c6e6-420f-a791-503cb0279663 service neutron] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:13.038590 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ee7541-c6e6-420f-a791-503cb0279663 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:13.047874 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-85ee7541-c6e6-420f-a791-503cb0279663 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:13.074976 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ee7541-c6e6-420f-a791-503cb0279663 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:13.076440 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ee7541-c6e6-420f-a791-503cb0279663 service neutron] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:13.083182 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 963/1894] 10.222.0.89 () {60 vars in 1153 bytes} [Thu Jul 28 17:45:13 2022] POST /identity/v3/projects => generated 375 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:45:13.084345 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c92ed7b-f3f9-46bf-89dd-265e308fae44 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:13.108834 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 932/1895] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:45:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:45:13.262156 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c92ed7b-f3f9-46bf-89dd-265e308fae44 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:13.282499 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c250c9-9c7f-44b1-b7f7-e9adeda21c0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:13.283285 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c250c9-9c7f-44b1-b7f7-e9adeda21c0d admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:13.283569 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c250c9-9c7f-44b1-b7f7-e9adeda21c0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:13.283839 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c250c9-9c7f-44b1-b7f7-e9adeda21c0d admin admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:13.284508 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c250c9-9c7f-44b1-b7f7-e9adeda21c0d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:13.286442 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c250c9-9c7f-44b1-b7f7-e9adeda21c0d admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:13.353829 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 964/1896] 10.222.0.89 () {60 vars in 1144 bytes} [Thu Jul 28 17:45:13 2022] POST /identity/v3/users => generated 288 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:45:13.359131 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ee7541-c6e6-420f-a791-503cb0279663 service neutron] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:13.393765 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ee7541-c6e6-420f-a791-503cb0279663 service neutron] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:13.418055 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb1fe80-e2c0-49db-ae00-671661f771a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:13.420577 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb1fe80-e2c0-49db-ae00-671661f771a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:13.421501 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb1fe80-e2c0-49db-ae00-671661f771a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:13.421751 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb1fe80-e2c0-49db-ae00-671661f771a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:13.422537 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb1fe80-e2c0-49db-ae00-671661f771a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:13.437705 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb1fe80-e2c0-49db-ae00-671661f771a4 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:13.487383 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 933/1897] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:45:13 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:45:13.488684 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c250c9-9c7f-44b1-b7f7-e9adeda21c0d admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:13.529002 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c250c9-9c7f-44b1-b7f7-e9adeda21c0d admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:13.564881 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a345ed-6712-4271-b7fb-cae7b8456915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:13.566333 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a345ed-6712-4271-b7fb-cae7b8456915 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:13.568318 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a345ed-6712-4271-b7fb-cae7b8456915 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:13.570127 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a345ed-6712-4271-b7fb-cae7b8456915 admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:13.571618 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a345ed-6712-4271-b7fb-cae7b8456915 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:13.574955 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a345ed-6712-4271-b7fb-cae7b8456915 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:13.641319 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb1fe80-e2c0-49db-ae00-671661f771a4 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:13.641928 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 965/1898] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:45:13 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:45:13.663948 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb1fe80-e2c0-49db-ae00-671661f771a4 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:13.675417 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844335df-73d7-4a4b-b988-032cadc3631e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:13.676419 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844335df-73d7-4a4b-b988-032cadc3631e admin admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:13.676838 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844335df-73d7-4a4b-b988-032cadc3631e admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:13.677286 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844335df-73d7-4a4b-b988-032cadc3631e admin admin] PATH_INFO: `/v3/projects/dccaf4ea0ccd43fdb2d8a7a83cf1e94b/users/d6c606ccda304c4084a7150d5bd34e81/roles/11f8731756fe4caca13efa61b71b11c5` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:13.720896 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844335df-73d7-4a4b-b988-032cadc3631e admin admin] RBAC: Authorizing `identity:create_grant(project_id=dccaf4ea0ccd43fdb2d8a7a83cf1e94b, user_id=d6c606ccda304c4084a7150d5bd34e81, role_id=11f8731756fe4caca13efa61b71b11c5)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:13.727542 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844335df-73d7-4a4b-b988-032cadc3631e admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:13.761550 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 934/1899] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:45:13 2022] PUT /identity/v3/projects/dccaf4ea0ccd43fdb2d8a7a83cf1e94b/users/d6c606ccda304c4084a7150d5bd34e81/roles/11f8731756fe4caca13efa61b71b11c5 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 17:45:13.773485 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d20c462-4c3f-4ecf-8133-8b62519ebe0f None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:13.773774 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d20c462-4c3f-4ecf-8133-8b62519ebe0f None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:13.774051 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d20c462-4c3f-4ecf-8133-8b62519ebe0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:13.812564 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-4d20c462-4c3f-4ecf-8133-8b62519ebe0f None None] MFA Rules not processed for user `d6c606ccda304c4084a7150d5bd34e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:45:13.812564 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-4d20c462-4c3f-4ecf-8133-8b62519ebe0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:13.859057 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:45:13.972863 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 966/1900] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:45:13 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:45:13.990436 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844335df-73d7-4a4b-b988-032cadc3631e admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:13.992202 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-844335df-73d7-4a4b-b988-032cadc3631e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:14.036513 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:45:14.073978 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844335df-73d7-4a4b-b988-032cadc3631e admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:14.080316 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1189ab-2866-4502-a17e-de06e02a8b3a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96d512d0c84240918dae6c4a3e8b7b79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:14.081000 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1189ab-2866-4502-a17e-de06e02a8b3a service cinder] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:14.081385 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1189ab-2866-4502-a17e-de06e02a8b3a service cinder] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:14.081715 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1189ab-2866-4502-a17e-de06e02a8b3a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:14.096589 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1189ab-2866-4502-a17e-de06e02a8b3a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:14.099551 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1189ab-2866-4502-a17e-de06e02a8b3a service cinder] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:14.112239 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 935/1901] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:45:13 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:45:14.192070 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d20c462-4c3f-4ecf-8133-8b62519ebe0f None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:14.195063 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-4d20c462-4c3f-4ecf-8133-8b62519ebe0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:14.306911 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d20c462-4c3f-4ecf-8133-8b62519ebe0f None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:14.317690 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd619c7-c4d5-4a38-84d9-c44e0df2c063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:14.318862 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd619c7-c4d5-4a38-84d9-c44e0df2c063 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:14.319308 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd619c7-c4d5-4a38-84d9-c44e0df2c063 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:14.319985 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd619c7-c4d5-4a38-84d9-c44e0df2c063 admin admin] PATH_INFO: `/v3/projects/e58049119104464e99609cf83730d2d8` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:14.321490 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd619c7-c4d5-4a38-84d9-c44e0df2c063 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e58049119104464e99609cf83730d2d8)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:14.323444 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd619c7-c4d5-4a38-84d9-c44e0df2c063 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:14.402115 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-dbd619c7-c4d5-4a38-84d9-c44e0df2c063 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e58049119104464e99609cf83730d2d8', 'request_id': 'req-dbd619c7-c4d5-4a38-84d9-c44e0df2c063'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:14.411178 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-dbd619c7-c4d5-4a38-84d9-c44e0df2c063 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e58049119104464e99609cf83730d2d8', 'request_id': 'req-dbd619c7-c4d5-4a38-84d9-c44e0df2c063'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:14.422538 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-dbd619c7-c4d5-4a38-84d9-c44e0df2c063 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e58049119104464e99609cf83730d2d8', 'request_id': 'req-dbd619c7-c4d5-4a38-84d9-c44e0df2c063'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:14.449465 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-dbd619c7-c4d5-4a38-84d9-c44e0df2c063 admin admin] The token cache is being invalidate because project e58049119104464e99609cf83730d2d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80324) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 17:45:14.450358 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-dbd619c7-c4d5-4a38-84d9-c44e0df2c063 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:14.451213 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-dbd619c7-c4d5-4a38-84d9-c44e0df2c063 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:14.453903 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 967/1902] 10.222.0.89 () {58 vars in 1233 bytes} [Thu Jul 28 17:45:14 2022] DELETE /identity/v3/projects/e58049119104464e99609cf83730d2d8 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:45:14.459424 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4450476b-fe7f-4f40-bec0-7ff8b6cb685b None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:14.459982 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4450476b-fe7f-4f40-bec0-7ff8b6cb685b None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:14.460387 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4450476b-fe7f-4f40-bec0-7ff8b6cb685b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:14.501674 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-4450476b-fe7f-4f40-bec0-7ff8b6cb685b None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:45:14.503893 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-4450476b-fe7f-4f40-bec0-7ff8b6cb685b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:14.511308 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd619c7-c4d5-4a38-84d9-c44e0df2c063 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:14.513603 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-dbd619c7-c4d5-4a38-84d9-c44e0df2c063 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:14.671527 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:45:14.710931 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd619c7-c4d5-4a38-84d9-c44e0df2c063 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:14.711531 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 936/1903] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:45:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:45:14.716938 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80784f92-2c8f-4e10-8052-fcf5a4563303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:14.718345 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80784f92-2c8f-4e10-8052-fcf5a4563303 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:14.718781 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80784f92-2c8f-4e10-8052-fcf5a4563303 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:14.719219 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80784f92-2c8f-4e10-8052-fcf5a4563303 admin admin] PATH_INFO: `/v3/projects/d333e36a18f74154a092b992d9ac79be` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:14.720623 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80784f92-2c8f-4e10-8052-fcf5a4563303 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d333e36a18f74154a092b992d9ac79be)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:14.721206 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeac86a-82f8-49d2-bcee-a7125b693f52 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:14.721586 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeac86a-82f8-49d2-bcee-a7125b693f52 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:14.721903 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeac86a-82f8-49d2-bcee-a7125b693f52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:14.723373 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80784f92-2c8f-4e10-8052-fcf5a4563303 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:14.746126 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-ddeac86a-82f8-49d2-bcee-a7125b693f52 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:45:14.747311 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-ddeac86a-82f8-49d2-bcee-a7125b693f52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:14.765199 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 937/1904] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:45:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:45:14.767502 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddeac86a-82f8-49d2-bcee-a7125b693f52 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:14.783618 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddeac86a-82f8-49d2-bcee-a7125b693f52 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:14.787519 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-80784f92-2c8f-4e10-8052-fcf5a4563303 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd333e36a18f74154a092b992d9ac79be', 'request_id': 'req-80784f92-2c8f-4e10-8052-fcf5a4563303'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:14.793848 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-80784f92-2c8f-4e10-8052-fcf5a4563303 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd333e36a18f74154a092b992d9ac79be', 'request_id': 'req-80784f92-2c8f-4e10-8052-fcf5a4563303'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:14.801440 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-80784f92-2c8f-4e10-8052-fcf5a4563303 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd333e36a18f74154a092b992d9ac79be', 'request_id': 'req-80784f92-2c8f-4e10-8052-fcf5a4563303'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:14.827837 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-80784f92-2c8f-4e10-8052-fcf5a4563303 admin admin] The token cache is being invalidate because project d333e36a18f74154a092b992d9ac79be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80324) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 17:45:14.827837 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-80784f92-2c8f-4e10-8052-fcf5a4563303 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:14.829142 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-80784f92-2c8f-4e10-8052-fcf5a4563303 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:14.835708 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 968/1905] 10.222.0.89 () {58 vars in 1233 bytes} [Thu Jul 28 17:45:14 2022] DELETE /identity/v3/projects/d333e36a18f74154a092b992d9ac79be => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:45:14.839714 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80784f92-2c8f-4e10-8052-fcf5a4563303 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:14.842135 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-80784f92-2c8f-4e10-8052-fcf5a4563303 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:14.926163 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd716294-5c12-483c-abe8-151b8a77a293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:14.926795 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd716294-5c12-483c-abe8-151b8a77a293 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:14.927071 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd716294-5c12-483c-abe8-151b8a77a293 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:14.927518 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd716294-5c12-483c-abe8-151b8a77a293 admin admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:14.928420 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd716294-5c12-483c-abe8-151b8a77a293 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:45:14.928530 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd716294-5c12-483c-abe8-151b8a77a293 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:14.930119 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd716294-5c12-483c-abe8-151b8a77a293 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:14.948565 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 938/1906] 10.222.0.89 () {58 vars in 1153 bytes} [Thu Jul 28 17:45:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:45:14.955277 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd716294-5c12-483c-abe8-151b8a77a293 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:14.958915 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-cd716294-5c12-483c-abe8-151b8a77a293 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:14.978299 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80784f92-2c8f-4e10-8052-fcf5a4563303 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:14.985290 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea098dd-7e91-45fe-af4e-cb81d6348e8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:14.986743 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea098dd-7e91-45fe-af4e-cb81d6348e8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:14.987135 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea098dd-7e91-45fe-af4e-cb81d6348e8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:14.987514 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea098dd-7e91-45fe-af4e-cb81d6348e8f admin admin] PATH_INFO: `/v3/users/abc1100721eb4316b7265bcdcb14b7a5` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:14.989663 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea098dd-7e91-45fe-af4e-cb81d6348e8f admin admin] RBAC: Authorizing `identity:delete_user(user_id=abc1100721eb4316b7265bcdcb14b7a5)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:14.992014 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea098dd-7e91-45fe-af4e-cb81d6348e8f admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:14.992773 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd716294-5c12-483c-abe8-151b8a77a293 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:14.998791 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589893e7-4fbc-4449-92e2-220186832448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:15.001134 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589893e7-4fbc-4449-92e2-220186832448 admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:15.001643 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589893e7-4fbc-4449-92e2-220186832448 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:15.002068 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589893e7-4fbc-4449-92e2-220186832448 admin admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:15.002910 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589893e7-4fbc-4449-92e2-220186832448 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:15.005340 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589893e7-4fbc-4449-92e2-220186832448 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:15.054476 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 939/1907] 10.222.0.89 () {60 vars in 1153 bytes} [Thu Jul 28 17:45:14 2022] POST /identity/v3/projects => generated 369 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 17:45:15.056279 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589893e7-4fbc-4449-92e2-220186832448 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:15.076882 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-2ea098dd-7e91-45fe-af4e-cb81d6348e8f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'abc1100721eb4316b7265bcdcb14b7a5'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:15.096986 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-2ea098dd-7e91-45fe-af4e-cb81d6348e8f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'abc1100721eb4316b7265bcdcb14b7a5'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:15.098383 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-2ea098dd-7e91-45fe-af4e-cb81d6348e8f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'abc1100721eb4316b7265bcdcb14b7a5'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:15.113105 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-2ea098dd-7e91-45fe-af4e-cb81d6348e8f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'abc1100721eb4316b7265bcdcb14b7a5'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:15.132456 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:45:15.150408 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:45:15.153412 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-2ea098dd-7e91-45fe-af4e-cb81d6348e8f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'abc1100721eb4316b7265bcdcb14b7a5'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:15.164781 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 969/1908] 10.222.0.89 () {58 vars in 1224 bytes} [Thu Jul 28 17:45:14 2022] DELETE /identity/v3/users/abc1100721eb4316b7265bcdcb14b7a5 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 17:45:15.167558 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589893e7-4fbc-4449-92e2-220186832448 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:15.269455 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ead278-42ce-42ca-b44f-8e88f8fbd2d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:15.270129 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ead278-42ce-42ca-b44f-8e88f8fbd2d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:15.270606 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ead278-42ce-42ca-b44f-8e88f8fbd2d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:15.271108 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ead278-42ce-42ca-b44f-8e88f8fbd2d3 admin admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:15.272207 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ead278-42ce-42ca-b44f-8e88f8fbd2d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:15.274801 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ead278-42ce-42ca-b44f-8e88f8fbd2d3 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:15.321575 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 940/1909] 10.222.0.89 () {60 vars in 1144 bytes} [Thu Jul 28 17:45:15 2022] POST /identity/v3/users => generated 285 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 17:45:15.324582 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea098dd-7e91-45fe-af4e-cb81d6348e8f admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:15.326082 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-2ea098dd-7e91-45fe-af4e-cb81d6348e8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:15.353437 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea098dd-7e91-45fe-af4e-cb81d6348e8f admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:15.361623 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2169fe03-71eb-48ee-a6ac-083363b10bca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:15.362410 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2169fe03-71eb-48ee-a6ac-083363b10bca admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:15.362791 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2169fe03-71eb-48ee-a6ac-083363b10bca admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:15.363412 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2169fe03-71eb-48ee-a6ac-083363b10bca admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:15.364237 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2169fe03-71eb-48ee-a6ac-083363b10bca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:15.366846 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2169fe03-71eb-48ee-a6ac-083363b10bca admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:15.381415 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 970/1910] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:45:15 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:45:15.386241 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ead278-42ce-42ca-b44f-8e88f8fbd2d3 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:15.401439 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ead278-42ce-42ca-b44f-8e88f8fbd2d3 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:15.418258 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c506f16-a54d-44e9-be90-f4f55c581912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:15.418894 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c506f16-a54d-44e9-be90-f4f55c581912 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:15.419170 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c506f16-a54d-44e9-be90-f4f55c581912 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:15.419558 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c506f16-a54d-44e9-be90-f4f55c581912 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:15.420136 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c506f16-a54d-44e9-be90-f4f55c581912 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:15.422132 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c506f16-a54d-44e9-be90-f4f55c581912 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:15.437244 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 941/1911] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:45:15 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:45:15.441281 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2169fe03-71eb-48ee-a6ac-083363b10bca admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:15.465783 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2169fe03-71eb-48ee-a6ac-083363b10bca admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:15.471460 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e528c6-a512-47ee-8306-e09059558b46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:15.472229 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e528c6-a512-47ee-8306-e09059558b46 admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:15.472577 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e528c6-a512-47ee-8306-e09059558b46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:15.472907 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e528c6-a512-47ee-8306-e09059558b46 admin admin] PATH_INFO: `/v3/projects/f7cb9aa09e5944469274f989c04cb778/users/f31c91844b174de4aec15e26f7aa3580/roles/11f8731756fe4caca13efa61b71b11c5` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:15.489446 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e528c6-a512-47ee-8306-e09059558b46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7cb9aa09e5944469274f989c04cb778, user_id=f31c91844b174de4aec15e26f7aa3580, role_id=11f8731756fe4caca13efa61b71b11c5)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:15.492194 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e528c6-a512-47ee-8306-e09059558b46 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:15.508369 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 971/1912] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:45:15 2022] PUT /identity/v3/projects/f7cb9aa09e5944469274f989c04cb778/users/f31c91844b174de4aec15e26f7aa3580/roles/11f8731756fe4caca13efa61b71b11c5 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 17:45:15.519439 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9fb650-54e6-4796-b44e-2d9cbcbf3cd2 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:15.519727 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9fb650-54e6-4796-b44e-2d9cbcbf3cd2 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:15.519998 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9fb650-54e6-4796-b44e-2d9cbcbf3cd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:15.551241 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-9d9fb650-54e6-4796-b44e-2d9cbcbf3cd2 None None] MFA Rules not processed for user `f31c91844b174de4aec15e26f7aa3580`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:45:15.552486 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-9d9fb650-54e6-4796-b44e-2d9cbcbf3cd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:15.556804 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e528c6-a512-47ee-8306-e09059558b46 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:15.558671 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-24e528c6-a512-47ee-8306-e09059558b46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:15.636182 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e528c6-a512-47ee-8306-e09059558b46 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:15.637976 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 942/1913] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:45:15 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:45:15.642556 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9fb650-54e6-4796-b44e-2d9cbcbf3cd2 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:15.644482 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f1f940-febd-4e3d-b121-430c20e1334a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:15.645167 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f1f940-febd-4e3d-b121-430c20e1334a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:15.645455 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f1f940-febd-4e3d-b121-430c20e1334a admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:15.645728 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f1f940-febd-4e3d-b121-430c20e1334a admin admin] PATH_INFO: `/v3/projects/7463ef96c10a45d1903b7c38e55bdb04` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:15.646909 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f1f940-febd-4e3d-b121-430c20e1334a admin admin] RBAC: Authorizing `identity:delete_project(project_id=7463ef96c10a45d1903b7c38e55bdb04)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:15.648852 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f1f940-febd-4e3d-b121-430c20e1334a admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:15.653260 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9fb650-54e6-4796-b44e-2d9cbcbf3cd2 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:15.661005 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2f5d39-ec20-4cb5-b21e-3314d4c3ecf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:15.661881 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2f5d39-ec20-4cb5-b21e-3314d4c3ecf2 admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:15.661984 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2f5d39-ec20-4cb5-b21e-3314d4c3ecf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:15.662235 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2f5d39-ec20-4cb5-b21e-3314d4c3ecf2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:15.663038 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2f5d39-ec20-4cb5-b21e-3314d4c3ecf2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:15.665174 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2f5d39-ec20-4cb5-b21e-3314d4c3ecf2 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:15.696588 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 943/1914] 10.222.0.89 () {60 vars in 1153 bytes} [Thu Jul 28 17:45:15 2022] POST /identity/v3/projects => generated 367 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 17:45:15.700611 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2f5d39-ec20-4cb5-b21e-3314d4c3ecf2 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:15.703071 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-25f1f940-febd-4e3d-b121-430c20e1334a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7463ef96c10a45d1903b7c38e55bdb04', 'request_id': 'req-25f1f940-febd-4e3d-b121-430c20e1334a'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:15.721764 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-25f1f940-febd-4e3d-b121-430c20e1334a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7463ef96c10a45d1903b7c38e55bdb04', 'request_id': 'req-25f1f940-febd-4e3d-b121-430c20e1334a'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:15.731182 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-25f1f940-febd-4e3d-b121-430c20e1334a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7463ef96c10a45d1903b7c38e55bdb04', 'request_id': 'req-25f1f940-febd-4e3d-b121-430c20e1334a'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:15.740336 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:45:15.749566 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-25f1f940-febd-4e3d-b121-430c20e1334a admin admin] The token cache is being invalidate because project 7463ef96c10a45d1903b7c38e55bdb04 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80324) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 17:45:15.750020 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-25f1f940-febd-4e3d-b121-430c20e1334a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:15.750831 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-25f1f940-febd-4e3d-b121-430c20e1334a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:15.752734 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 972/1915] 10.222.0.89 () {58 vars in 1233 bytes} [Thu Jul 28 17:45:15 2022] DELETE /identity/v3/projects/7463ef96c10a45d1903b7c38e55bdb04 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:45:15.758705 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeeefa4-fe5c-4829-b4a0-965b9abee9d0 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:15.759119 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeeefa4-fe5c-4829-b4a0-965b9abee9d0 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:15.759466 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeeefa4-fe5c-4829-b4a0-965b9abee9d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:15.781149 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2f5d39-ec20-4cb5-b21e-3314d4c3ecf2 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:15.782932 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-dfeeefa4-fe5c-4829-b4a0-965b9abee9d0 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:45:15.784596 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-dfeeefa4-fe5c-4829-b4a0-965b9abee9d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:15.786247 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c937d2-db86-4177-8b42-6aeda264134b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:15.786975 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c937d2-db86-4177-8b42-6aeda264134b admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:15.787390 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c937d2-db86-4177-8b42-6aeda264134b admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:15.787660 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c937d2-db86-4177-8b42-6aeda264134b admin admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:15.788402 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c937d2-db86-4177-8b42-6aeda264134b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:15.790438 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c937d2-db86-4177-8b42-6aeda264134b admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:15.794438 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 973/1916] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:45:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:45:15.803003 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384d5999-98d5-42d3-9049-f5b8d6a9905b None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:15.803290 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384d5999-98d5-42d3-9049-f5b8d6a9905b None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:15.803562 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384d5999-98d5-42d3-9049-f5b8d6a9905b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:15.830301 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-384d5999-98d5-42d3-9049-f5b8d6a9905b None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:45:15.831578 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-384d5999-98d5-42d3-9049-f5b8d6a9905b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:15.839757 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 944/1917] 10.222.0.89 () {60 vars in 1144 bytes} [Thu Jul 28 17:45:15 2022] POST /identity/v3/users => generated 290 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:45:15.843909 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c937d2-db86-4177-8b42-6aeda264134b admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:15.845885 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-e1c937d2-db86-4177-8b42-6aeda264134b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:15.846280 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 974/1918] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:45:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:45:15.850078 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384d5999-98d5-42d3-9049-f5b8d6a9905b None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:15.861436 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c937d2-db86-4177-8b42-6aeda264134b admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:15.869376 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091413ce-aa66-4307-8370-c393cc3dd508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:15.870760 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091413ce-aa66-4307-8370-c393cc3dd508 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:15.871096 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091413ce-aa66-4307-8370-c393cc3dd508 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:15.871522 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091413ce-aa66-4307-8370-c393cc3dd508 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:15.872492 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091413ce-aa66-4307-8370-c393cc3dd508 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:15.873534 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384d5999-98d5-42d3-9049-f5b8d6a9905b None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:15.876273 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091413ce-aa66-4307-8370-c393cc3dd508 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:15.882780 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754984b1-c92a-43ce-becc-2ca7d188fbcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:15.883729 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754984b1-c92a-43ce-becc-2ca7d188fbcf admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:15.884261 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754984b1-c92a-43ce-becc-2ca7d188fbcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:15.885003 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754984b1-c92a-43ce-becc-2ca7d188fbcf admin admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:15.886927 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754984b1-c92a-43ce-becc-2ca7d188fbcf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:45:15.887331 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754984b1-c92a-43ce-becc-2ca7d188fbcf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:15.889048 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754984b1-c92a-43ce-becc-2ca7d188fbcf admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:15.894508 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 945/1919] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:45:15 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:45:15.904622 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 975/1920] 10.222.0.89 () {58 vars in 1153 bytes} [Thu Jul 28 17:45:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:45:15.909146 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091413ce-aa66-4307-8370-c393cc3dd508 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:15.910026 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754984b1-c92a-43ce-becc-2ca7d188fbcf admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:15.926757 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091413ce-aa66-4307-8370-c393cc3dd508 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:15.927683 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754984b1-c92a-43ce-becc-2ca7d188fbcf admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:15.933968 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5404043c-1bd3-4004-b6d9-2add700d0d2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:15.934695 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5404043c-1bd3-4004-b6d9-2add700d0d2d admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:15.935097 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5404043c-1bd3-4004-b6d9-2add700d0d2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:15.936035 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5404043c-1bd3-4004-b6d9-2add700d0d2d admin admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:15.937141 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5404043c-1bd3-4004-b6d9-2add700d0d2d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:15.939826 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5404043c-1bd3-4004-b6d9-2add700d0d2d admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:15.945370 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edb8501-1f57-4a18-a0cf-0b7fdd857a46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:15.946208 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edb8501-1f57-4a18-a0cf-0b7fdd857a46 admin admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:15.946369 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edb8501-1f57-4a18-a0cf-0b7fdd857a46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:15.946674 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edb8501-1f57-4a18-a0cf-0b7fdd857a46 admin admin] PATH_INFO: `/v3/projects/99ff43f7933040ba8a1b9de924b132ab/users/9b32cca9d4564991bf03f21a5661fd39/roles/73e457231a824ea98c28d22610be640c` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:15.991222 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edb8501-1f57-4a18-a0cf-0b7fdd857a46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99ff43f7933040ba8a1b9de924b132ab, user_id=9b32cca9d4564991bf03f21a5661fd39, role_id=73e457231a824ea98c28d22610be640c)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:15.994777 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edb8501-1f57-4a18-a0cf-0b7fdd857a46 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:15.998892 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 976/1921] 10.222.0.89 () {60 vars in 1153 bytes} [Thu Jul 28 17:45:15 2022] POST /identity/v3/projects => generated 375 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:45:16.000018 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5404043c-1bd3-4004-b6d9-2add700d0d2d admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:16.007754 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:45:16.019153 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 946/1922] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:45:15 2022] PUT /identity/v3/projects/99ff43f7933040ba8a1b9de924b132ab/users/9b32cca9d4564991bf03f21a5661fd39/roles/73e457231a824ea98c28d22610be640c => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:45:16.023501 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edb8501-1f57-4a18-a0cf-0b7fdd857a46 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:16.093537 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5404043c-1bd3-4004-b6d9-2add700d0d2d admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:16.095202 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edb8501-1f57-4a18-a0cf-0b7fdd857a46 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:16.099804 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea1bf5c-cc98-48a3-bb88-14f2a206488d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:16.100509 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea1bf5c-cc98-48a3-bb88-14f2a206488d admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:16.100861 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea1bf5c-cc98-48a3-bb88-14f2a206488d admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:16.101257 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea1bf5c-cc98-48a3-bb88-14f2a206488d admin admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:16.101687 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2bbac8-089c-4062-8843-587eb499ccb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:16.102102 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea1bf5c-cc98-48a3-bb88-14f2a206488d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:16.102313 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2bbac8-089c-4062-8843-587eb499ccb6 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:16.102600 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2bbac8-089c-4062-8843-587eb499ccb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:16.102876 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2bbac8-089c-4062-8843-587eb499ccb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:16.103581 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2bbac8-089c-4062-8843-587eb499ccb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:16.104542 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea1bf5c-cc98-48a3-bb88-14f2a206488d admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:16.107195 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2bbac8-089c-4062-8843-587eb499ccb6 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:16.119195 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 947/1923] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:45:16 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:45:16.123502 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2bbac8-089c-4062-8843-587eb499ccb6 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:16.133700 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 977/1924] 10.222.0.89 () {60 vars in 1144 bytes} [Thu Jul 28 17:45:15 2022] POST /identity/v3/users => generated 288 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 17:45:16.135028 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2bbac8-089c-4062-8843-587eb499ccb6 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:16.135151 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea1bf5c-cc98-48a3-bb88-14f2a206488d admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:16.141349 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9408ec92-2201-4331-ac66-b592fe4a2ad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:16.142174 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9408ec92-2201-4331-ac66-b592fe4a2ad8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:16.142558 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9408ec92-2201-4331-ac66-b592fe4a2ad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:16.142937 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9408ec92-2201-4331-ac66-b592fe4a2ad8 admin admin] PATH_INFO: `/v3/projects/99ff43f7933040ba8a1b9de924b132ab/users/9b32cca9d4564991bf03f21a5661fd39/roles/73e457231a824ea98c28d22610be640c` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:16.145189 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9408ec92-2201-4331-ac66-b592fe4a2ad8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99ff43f7933040ba8a1b9de924b132ab, user_id=9b32cca9d4564991bf03f21a5661fd39, role_id=73e457231a824ea98c28d22610be640c)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:16.147978 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea1bf5c-cc98-48a3-bb88-14f2a206488d admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:16.148424 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9408ec92-2201-4331-ac66-b592fe4a2ad8 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:16.152701 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b28bc7-05b6-49d0-80bf-7b0d89616dcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:16.154089 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b28bc7-05b6-49d0-80bf-7b0d89616dcc admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:16.154374 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b28bc7-05b6-49d0-80bf-7b0d89616dcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:16.154664 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b28bc7-05b6-49d0-80bf-7b0d89616dcc admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:16.156330 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b28bc7-05b6-49d0-80bf-7b0d89616dcc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:16.158112 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b28bc7-05b6-49d0-80bf-7b0d89616dcc admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:16.165758 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 948/1925] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:45:16 2022] PUT /identity/v3/projects/99ff43f7933040ba8a1b9de924b132ab/users/9b32cca9d4564991bf03f21a5661fd39/roles/73e457231a824ea98c28d22610be640c => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 17:45:16.174647 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 978/1926] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:45:16 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:45:16.175776 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b28bc7-05b6-49d0-80bf-7b0d89616dcc admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:16.175925 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4371190a-32a9-45c8-aed4-2c42caf4564c None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:16.176140 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4371190a-32a9-45c8-aed4-2c42caf4564c None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:16.176486 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4371190a-32a9-45c8-aed4-2c42caf4564c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:16.193303 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b28bc7-05b6-49d0-80bf-7b0d89616dcc admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:16.212530 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-4371190a-32a9-45c8-aed4-2c42caf4564c None None] MFA Rules not processed for user `9b32cca9d4564991bf03f21a5661fd39`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:45:16.215047 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-4371190a-32a9-45c8-aed4-2c42caf4564c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:16.268854 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f177ad-35e7-424b-bc9a-c10dd8cb76ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:16.269524 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f177ad-35e7-424b-bc9a-c10dd8cb76ae admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:16.269813 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f177ad-35e7-424b-bc9a-c10dd8cb76ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:16.270093 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f177ad-35e7-424b-bc9a-c10dd8cb76ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:16.270852 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f177ad-35e7-424b-bc9a-c10dd8cb76ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:16.272830 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f177ad-35e7-424b-bc9a-c10dd8cb76ae admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:16.285295 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 979/1927] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:45:16 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 17:45:16.290053 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f177ad-35e7-424b-bc9a-c10dd8cb76ae admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:16.301504 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f177ad-35e7-424b-bc9a-c10dd8cb76ae admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:16.306602 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e54e22a-a273-40c8-9ae6-8fc417647bde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:16.307431 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e54e22a-a273-40c8-9ae6-8fc417647bde admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:16.307717 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e54e22a-a273-40c8-9ae6-8fc417647bde admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:16.307998 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e54e22a-a273-40c8-9ae6-8fc417647bde admin admin] PATH_INFO: `/v3/projects/e36277abfdd146a4beaa242fc6d0f3cf/users/1c8fae03ec514f54ad5e0d14a316bcc8/roles/11f8731756fe4caca13efa61b71b11c5` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:16.321965 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:45:16.324409 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e54e22a-a273-40c8-9ae6-8fc417647bde admin admin] RBAC: Authorizing `identity:create_grant(project_id=e36277abfdd146a4beaa242fc6d0f3cf, user_id=1c8fae03ec514f54ad5e0d14a316bcc8, role_id=11f8731756fe4caca13efa61b71b11c5)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:16.328875 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e54e22a-a273-40c8-9ae6-8fc417647bde admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:16.335723 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 949/1928] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:45:16 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:45:16.348696 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4371190a-32a9-45c8-aed4-2c42caf4564c None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:16.351117 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-4371190a-32a9-45c8-aed4-2c42caf4564c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:16.356798 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 980/1929] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:45:16 2022] PUT /identity/v3/projects/e36277abfdd146a4beaa242fc6d0f3cf/users/1c8fae03ec514f54ad5e0d14a316bcc8/roles/11f8731756fe4caca13efa61b71b11c5 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:45:16.370562 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03174903-8dad-4972-ad5c-b8544e5ff2f6 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:16.370933 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03174903-8dad-4972-ad5c-b8544e5ff2f6 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:16.371279 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03174903-8dad-4972-ad5c-b8544e5ff2f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:16.406104 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-03174903-8dad-4972-ad5c-b8544e5ff2f6 None None] MFA Rules not processed for user `1c8fae03ec514f54ad5e0d14a316bcc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:45:16.407778 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-03174903-8dad-4972-ad5c-b8544e5ff2f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:16.422128 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4371190a-32a9-45c8-aed4-2c42caf4564c None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:16.461411 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:45:16.478656 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6273f1d2-89b4-4633-802d-a18749716f6c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96d512d0c84240918dae6c4a3e8b7b79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:16.479510 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6273f1d2-89b4-4633-802d-a18749716f6c service cinder] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:16.479794 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6273f1d2-89b4-4633-802d-a18749716f6c service cinder] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:16.480283 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6273f1d2-89b4-4633-802d-a18749716f6c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:16.493949 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 981/1930] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:45:16 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 17:45:16.561264 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03174903-8dad-4972-ad5c-b8544e5ff2f6 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:16.563522 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-03174903-8dad-4972-ad5c-b8544e5ff2f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:16.568348 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6273f1d2-89b4-4633-802d-a18749716f6c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:16.570395 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6273f1d2-89b4-4633-802d-a18749716f6c service cinder] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:16.576798 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03174903-8dad-4972-ad5c-b8544e5ff2f6 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:16.582242 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0963db8a-bdfa-4d59-976d-fea870db55f9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96d512d0c84240918dae6c4a3e8b7b79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:16.582906 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0963db8a-bdfa-4d59-976d-fea870db55f9 service cinder] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:16.585226 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0963db8a-bdfa-4d59-976d-fea870db55f9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:16.585226 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0963db8a-bdfa-4d59-976d-fea870db55f9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:16.585451 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 950/1931] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:45:16 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:45:16.603266 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0963db8a-bdfa-4d59-976d-fea870db55f9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:16.605062 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0963db8a-bdfa-4d59-976d-fea870db55f9 service cinder] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:16.614881 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 982/1932] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:45:16 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:45:18.141107 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6273f1d2-89b4-4633-802d-a18749716f6c service cinder] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:18.203470 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6273f1d2-89b4-4633-802d-a18749716f6c service cinder] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:18.207987 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234be699-7919-4ae8-8042-62dd6b77c68d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:18.208622 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234be699-7919-4ae8-8042-62dd6b77c68d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:18.208903 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234be699-7919-4ae8-8042-62dd6b77c68d admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:18.209240 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234be699-7919-4ae8-8042-62dd6b77c68d admin admin] PATH_INFO: `/v3/users/f31c91844b174de4aec15e26f7aa3580` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:18.210410 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234be699-7919-4ae8-8042-62dd6b77c68d admin admin] RBAC: Authorizing `identity:delete_user(user_id=f31c91844b174de4aec15e26f7aa3580)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:18.212434 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234be699-7919-4ae8-8042-62dd6b77c68d admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:18.214026 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:45:18.263485 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-234be699-7919-4ae8-8042-62dd6b77c68d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f31c91844b174de4aec15e26f7aa3580'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:18.270964 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-234be699-7919-4ae8-8042-62dd6b77c68d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f31c91844b174de4aec15e26f7aa3580'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:18.279213 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-234be699-7919-4ae8-8042-62dd6b77c68d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f31c91844b174de4aec15e26f7aa3580'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:18.279861 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-234be699-7919-4ae8-8042-62dd6b77c68d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f31c91844b174de4aec15e26f7aa3580'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:18.281667 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-234be699-7919-4ae8-8042-62dd6b77c68d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f31c91844b174de4aec15e26f7aa3580'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:18.299751 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 951/1933] 10.222.0.89 () {58 vars in 1224 bytes} [Thu Jul 28 17:45:18 2022] DELETE /identity/v3/users/f31c91844b174de4aec15e26f7aa3580 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:45:18.304143 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0d0278-e94c-4c10-b5d9-58e5da2524eb None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:18.304606 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0d0278-e94c-4c10-b5d9-58e5da2524eb None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:18.304717 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0d0278-e94c-4c10-b5d9-58e5da2524eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:18.323988 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-1c0d0278-e94c-4c10-b5d9-58e5da2524eb None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:45:18.325113 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-1c0d0278-e94c-4c10-b5d9-58e5da2524eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:18.397306 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 983/1934] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:45:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:45:18.404230 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234be699-7919-4ae8-8042-62dd6b77c68d admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:18.406308 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-234be699-7919-4ae8-8042-62dd6b77c68d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:18.465595 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234be699-7919-4ae8-8042-62dd6b77c68d admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:18.469560 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116ebb29-c4d6-482e-88d3-43882491c29a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76817cd541294331b0023069092e8e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:18.470191 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116ebb29-c4d6-482e-88d3-43882491c29a service neutron] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:18.470477 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116ebb29-c4d6-482e-88d3-43882491c29a service neutron] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:18.470748 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116ebb29-c4d6-482e-88d3-43882491c29a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:18.480289 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116ebb29-c4d6-482e-88d3-43882491c29a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:18.481804 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116ebb29-c4d6-482e-88d3-43882491c29a service neutron] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:18.492415 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 952/1935] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:45:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:45:18.924857 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0d0278-e94c-4c10-b5d9-58e5da2524eb None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:18.927762 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-1c0d0278-e94c-4c10-b5d9-58e5da2524eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:18.942767 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0d0278-e94c-4c10-b5d9-58e5da2524eb None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:18.948950 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a68ce5-cd67-47d7-a68d-d42cdda33dc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:18.949789 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a68ce5-cd67-47d7-a68d-d42cdda33dc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:18.950078 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a68ce5-cd67-47d7-a68d-d42cdda33dc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:18.950349 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a68ce5-cd67-47d7-a68d-d42cdda33dc5 admin admin] PATH_INFO: `/v3/projects/f7cb9aa09e5944469274f989c04cb778` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:18.951702 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a68ce5-cd67-47d7-a68d-d42cdda33dc5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7cb9aa09e5944469274f989c04cb778)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:18.954119 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a68ce5-cd67-47d7-a68d-d42cdda33dc5 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:19.043652 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-f5a68ce5-cd67-47d7-a68d-d42cdda33dc5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7cb9aa09e5944469274f989c04cb778', 'request_id': 'req-f5a68ce5-cd67-47d7-a68d-d42cdda33dc5'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:19.045134 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-f5a68ce5-cd67-47d7-a68d-d42cdda33dc5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7cb9aa09e5944469274f989c04cb778', 'request_id': 'req-f5a68ce5-cd67-47d7-a68d-d42cdda33dc5'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:19.053683 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-f5a68ce5-cd67-47d7-a68d-d42cdda33dc5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7cb9aa09e5944469274f989c04cb778', 'request_id': 'req-f5a68ce5-cd67-47d7-a68d-d42cdda33dc5'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:19.066842 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-f5a68ce5-cd67-47d7-a68d-d42cdda33dc5 admin admin] The token cache is being invalidate because project f7cb9aa09e5944469274f989c04cb778 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80324) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 17:45:19.067136 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-f5a68ce5-cd67-47d7-a68d-d42cdda33dc5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:19.067989 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-f5a68ce5-cd67-47d7-a68d-d42cdda33dc5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:19.069610 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 984/1936] 10.222.0.89 () {58 vars in 1233 bytes} [Thu Jul 28 17:45:18 2022] DELETE /identity/v3/projects/f7cb9aa09e5944469274f989c04cb778 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:45:19.073734 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116ebb29-c4d6-482e-88d3-43882491c29a service neutron] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:19.085296 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116ebb29-c4d6-482e-88d3-43882491c29a service neutron] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:19.159509 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdffce5-e124-4be0-9b32-17caa47e3d5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:19.160189 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdffce5-e124-4be0-9b32-17caa47e3d5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:19.160476 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdffce5-e124-4be0-9b32-17caa47e3d5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:19.160749 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdffce5-e124-4be0-9b32-17caa47e3d5d admin admin] PATH_INFO: `/v3/users/9b32cca9d4564991bf03f21a5661fd39` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:19.162136 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdffce5-e124-4be0-9b32-17caa47e3d5d admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b32cca9d4564991bf03f21a5661fd39)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:19.164043 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdffce5-e124-4be0-9b32-17caa47e3d5d admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:19.189972 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:45:19.216062 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-2cdffce5-e124-4be0-9b32-17caa47e3d5d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b32cca9d4564991bf03f21a5661fd39'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:19.224903 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-2cdffce5-e124-4be0-9b32-17caa47e3d5d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b32cca9d4564991bf03f21a5661fd39'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:19.233585 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-2cdffce5-e124-4be0-9b32-17caa47e3d5d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b32cca9d4564991bf03f21a5661fd39'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:19.234223 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-2cdffce5-e124-4be0-9b32-17caa47e3d5d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b32cca9d4564991bf03f21a5661fd39'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:19.234838 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-2cdffce5-e124-4be0-9b32-17caa47e3d5d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b32cca9d4564991bf03f21a5661fd39'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:19.259310 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 953/1937] 10.222.0.89 () {58 vars in 1224 bytes} [Thu Jul 28 17:45:19 2022] DELETE /identity/v3/users/9b32cca9d4564991bf03f21a5661fd39 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:45:19.588065 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a68ce5-cd67-47d7-a68d-d42cdda33dc5 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:19.591058 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-f5a68ce5-cd67-47d7-a68d-d42cdda33dc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:19.611897 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a68ce5-cd67-47d7-a68d-d42cdda33dc5 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:19.691611 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:45:19.706229 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb04340-e744-43d5-9052-6154642c9ca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:19.707427 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb04340-e744-43d5-9052-6154642c9ca5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:19.708089 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb04340-e744-43d5-9052-6154642c9ca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:19.708451 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb04340-e744-43d5-9052-6154642c9ca5 admin admin] PATH_INFO: `/v3/projects/99ff43f7933040ba8a1b9de924b132ab` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:19.710337 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb04340-e744-43d5-9052-6154642c9ca5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=99ff43f7933040ba8a1b9de924b132ab)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:19.712819 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb04340-e744-43d5-9052-6154642c9ca5 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:19.762953 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-adb04340-e744-43d5-9052-6154642c9ca5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99ff43f7933040ba8a1b9de924b132ab', 'request_id': 'req-adb04340-e744-43d5-9052-6154642c9ca5'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:19.768948 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-adb04340-e744-43d5-9052-6154642c9ca5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99ff43f7933040ba8a1b9de924b132ab', 'request_id': 'req-adb04340-e744-43d5-9052-6154642c9ca5'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:19.775602 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-adb04340-e744-43d5-9052-6154642c9ca5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99ff43f7933040ba8a1b9de924b132ab', 'request_id': 'req-adb04340-e744-43d5-9052-6154642c9ca5'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:19.786939 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae17e77b-a78e-4f7b-a7c6-228db18ff938 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:19.787257 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae17e77b-a78e-4f7b-a7c6-228db18ff938 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:19.787535 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae17e77b-a78e-4f7b-a7c6-228db18ff938 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:19.788899 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 954/1938] 10.222.0.89 () {54 vars in 921 bytes} [Thu Jul 28 17:45:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 17:45:19.793365 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-adb04340-e744-43d5-9052-6154642c9ca5 admin admin] The token cache is being invalidate because project 99ff43f7933040ba8a1b9de924b132ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80324) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 17:45:19.793656 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-adb04340-e744-43d5-9052-6154642c9ca5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:19.794337 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-adb04340-e744-43d5-9052-6154642c9ca5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:19.795906 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 985/1939] 10.222.0.89 () {58 vars in 1233 bytes} [Thu Jul 28 17:45:19 2022] DELETE /identity/v3/projects/99ff43f7933040ba8a1b9de924b132ab => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:45:19.801972 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4f20d4-fc5c-488a-a4db-16ff78109ba0 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:19.802258 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4f20d4-fc5c-488a-a4db-16ff78109ba0 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:19.802536 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4f20d4-fc5c-488a-a4db-16ff78109ba0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:19.805353 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98704110-2de0-465c-9a68-5d630a51157c None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:19.805744 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98704110-2de0-465c-9a68-5d630a51157c None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:19.806071 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98704110-2de0-465c-9a68-5d630a51157c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:19.826862 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-8c4f20d4-fc5c-488a-a4db-16ff78109ba0 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:45:19.828791 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-8c4f20d4-fc5c-488a-a4db-16ff78109ba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:19.838004 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-98704110-2de0-465c-9a68-5d630a51157c None None] MFA Rules not processed for user `d10b49b29df94404a71f43273839c252`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:45:19.839312 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-98704110-2de0-465c-9a68-5d630a51157c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:19.938896 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 955/1940] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:45:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 17:45:19.940607 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c548af86-cc34-44cc-8416-53cc07728f3a None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:19.940964 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c548af86-cc34-44cc-8416-53cc07728f3a None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:19.941392 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c548af86-cc34-44cc-8416-53cc07728f3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:19.949896 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 986/1941] 10.222.0.89 () {58 vars in 1018 bytes} [Thu Jul 28 17:45:19 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 17:45:19.963864 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-c548af86-cc34-44cc-8416-53cc07728f3a None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:45:19.965550 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-c548af86-cc34-44cc-8416-53cc07728f3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:19.970114 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98704110-2de0-465c-9a68-5d630a51157c None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:19.971704 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-98704110-2de0-465c-9a68-5d630a51157c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:19.981591 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 956/1942] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:45:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:45:19.989035 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c548af86-cc34-44cc-8416-53cc07728f3a None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:19.990391 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98704110-2de0-465c-9a68-5d630a51157c None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:20.003558 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c548af86-cc34-44cc-8416-53cc07728f3a None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:20.004579 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3450c9e4-d2bc-4b99-b8d3-58922054b87b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd10b49b29df94404a71f43273839c252', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:20.005611 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3450c9e4-d2bc-4b99-b8d3-58922054b87b service nova] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:20.006559 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3450c9e4-d2bc-4b99-b8d3-58922054b87b service nova] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:20.006559 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3450c9e4-d2bc-4b99-b8d3-58922054b87b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:20.019357 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34859e3a-3b7f-470d-952c-99a7ae4b873c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:20.020390 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34859e3a-3b7f-470d-952c-99a7ae4b873c admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:20.020811 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34859e3a-3b7f-470d-952c-99a7ae4b873c admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:20.021311 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34859e3a-3b7f-470d-952c-99a7ae4b873c admin admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:20.022171 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34859e3a-3b7f-470d-952c-99a7ae4b873c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:45:20.022623 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34859e3a-3b7f-470d-952c-99a7ae4b873c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:20.024942 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34859e3a-3b7f-470d-952c-99a7ae4b873c admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:20.031250 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3450c9e4-d2bc-4b99-b8d3-58922054b87b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:20.039962 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3450c9e4-d2bc-4b99-b8d3-58922054b87b service nova] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:20.052622 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 957/1943] 10.222.0.89 () {58 vars in 1153 bytes} [Thu Jul 28 17:45:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:45:20.054118 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34859e3a-3b7f-470d-952c-99a7ae4b873c admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:20.068730 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34859e3a-3b7f-470d-952c-99a7ae4b873c admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:20.071770 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 987/1944] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:45:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:45:20.076395 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc50c209-d77c-4e39-a480-e77b11fcd532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:20.076712 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc50c209-d77c-4e39-a480-e77b11fcd532 admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:20.077487 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc50c209-d77c-4e39-a480-e77b11fcd532 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:20.077814 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc50c209-d77c-4e39-a480-e77b11fcd532 admin admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:20.078702 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc50c209-d77c-4e39-a480-e77b11fcd532 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:20.081661 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc50c209-d77c-4e39-a480-e77b11fcd532 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:20.120910 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 958/1945] 10.222.0.89 () {60 vars in 1153 bytes} [Thu Jul 28 17:45:20 2022] POST /identity/v3/projects => generated 379 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 17:45:20.125397 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3450c9e4-d2bc-4b99-b8d3-58922054b87b service nova] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:20.137418 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3450c9e4-d2bc-4b99-b8d3-58922054b87b service nova] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:20.253304 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce072ac-fdc9-4501-bb09-24c4db0933b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:20.253931 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce072ac-fdc9-4501-bb09-24c4db0933b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:20.254216 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce072ac-fdc9-4501-bb09-24c4db0933b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:20.254489 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce072ac-fdc9-4501-bb09-24c4db0933b4 admin admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:20.255173 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce072ac-fdc9-4501-bb09-24c4db0933b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:20.257094 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce072ac-fdc9-4501-bb09-24c4db0933b4 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:20.267349 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:45:20.304219 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc50c209-d77c-4e39-a480-e77b11fcd532 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:20.306844 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 988/1946] 10.222.0.89 () {60 vars in 1144 bytes} [Thu Jul 28 17:45:20 2022] POST /identity/v3/users => generated 290 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:45:20.328279 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc50c209-d77c-4e39-a480-e77b11fcd532 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:20.336366 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd536ab-d1ea-407f-9fc8-bad109c52b47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:20.337345 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd536ab-d1ea-407f-9fc8-bad109c52b47 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:20.338568 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd536ab-d1ea-407f-9fc8-bad109c52b47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:20.339060 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd536ab-d1ea-407f-9fc8-bad109c52b47 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:20.339977 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd536ab-d1ea-407f-9fc8-bad109c52b47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:20.342285 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd536ab-d1ea-407f-9fc8-bad109c52b47 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:20.356819 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 959/1947] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:45:20 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:45:20.361710 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce072ac-fdc9-4501-bb09-24c4db0933b4 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:20.379705 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce072ac-fdc9-4501-bb09-24c4db0933b4 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:20.386645 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d51e78-b027-4653-8a8f-c02fc040446e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:20.386993 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d51e78-b027-4653-8a8f-c02fc040446e admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:20.387373 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d51e78-b027-4653-8a8f-c02fc040446e admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:20.387749 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d51e78-b027-4653-8a8f-c02fc040446e admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:20.388983 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d51e78-b027-4653-8a8f-c02fc040446e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:20.391858 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d51e78-b027-4653-8a8f-c02fc040446e admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:20.405839 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 989/1948] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:45:20 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 17:45:20.407651 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd536ab-d1ea-407f-9fc8-bad109c52b47 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:20.418915 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd536ab-d1ea-407f-9fc8-bad109c52b47 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:20.424301 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d47806-68d3-4a83-b812-ab3f998f425f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:20.425253 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d47806-68d3-4a83-b812-ab3f998f425f admin admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:20.425691 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d47806-68d3-4a83-b812-ab3f998f425f admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:20.425998 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d47806-68d3-4a83-b812-ab3f998f425f admin admin] PATH_INFO: `/v3/projects/15da1db32f2d45a89d2dd022a05b9e4e/users/1423e3402c9246edaa0378b048372c1f/roles/11f8731756fe4caca13efa61b71b11c5` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:20.452516 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d47806-68d3-4a83-b812-ab3f998f425f admin admin] RBAC: Authorizing `identity:create_grant(project_id=15da1db32f2d45a89d2dd022a05b9e4e, user_id=1423e3402c9246edaa0378b048372c1f, role_id=11f8731756fe4caca13efa61b71b11c5)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:20.455466 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d47806-68d3-4a83-b812-ab3f998f425f admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:20.471650 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 960/1949] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:45:20 2022] PUT /identity/v3/projects/15da1db32f2d45a89d2dd022a05b9e4e/users/1423e3402c9246edaa0378b048372c1f/roles/11f8731756fe4caca13efa61b71b11c5 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:45:20.488807 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6510a7d2-62e6-40be-9919-108e503a47c0 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:20.489184 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6510a7d2-62e6-40be-9919-108e503a47c0 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:20.489486 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6510a7d2-62e6-40be-9919-108e503a47c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:20.520098 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-6510a7d2-62e6-40be-9919-108e503a47c0 None None] MFA Rules not processed for user `1423e3402c9246edaa0378b048372c1f`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:45:20.521434 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-6510a7d2-62e6-40be-9919-108e503a47c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:20.600207 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 990/1950] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:45:20 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:45:20.604771 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d47806-68d3-4a83-b812-ab3f998f425f admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:20.637135 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=91,keystone:DELETE=6,keystone:INSERT=3 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:45:20.701924 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d47806-68d3-4a83-b812-ab3f998f425f admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:20.707404 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae7eb9b-2800-4812-a87f-8f1934d44790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:20.708184 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae7eb9b-2800-4812-a87f-8f1934d44790 admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:20.708504 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae7eb9b-2800-4812-a87f-8f1934d44790 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:20.709035 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae7eb9b-2800-4812-a87f-8f1934d44790 admin admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:20.709974 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae7eb9b-2800-4812-a87f-8f1934d44790 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:20.712912 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae7eb9b-2800-4812-a87f-8f1934d44790 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:20.749218 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 961/1951] 10.222.0.89 () {60 vars in 1153 bytes} [Thu Jul 28 17:45:20 2022] POST /identity/v3/projects => generated 379 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:45:20.751568 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6510a7d2-62e6-40be-9919-108e503a47c0 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:20.764465 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6510a7d2-62e6-40be-9919-108e503a47c0 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:20.859655 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e05508-70cc-4c52-bc59-18c11ae03947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:20.860806 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e05508-70cc-4c52-bc59-18c11ae03947 admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:20.861775 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e05508-70cc-4c52-bc59-18c11ae03947 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:20.862219 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e05508-70cc-4c52-bc59-18c11ae03947 admin admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:20.863044 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e05508-70cc-4c52-bc59-18c11ae03947 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:20.865197 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e05508-70cc-4c52-bc59-18c11ae03947 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:20.895161 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 991/1952] 10.222.0.89 () {60 vars in 1144 bytes} [Thu Jul 28 17:45:20 2022] POST /identity/v3/users => generated 290 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:45:20.899121 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae7eb9b-2800-4812-a87f-8f1934d44790 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:20.911403 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae7eb9b-2800-4812-a87f-8f1934d44790 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:20.919452 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e80c44-c782-48e8-b12f-441700d27769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:20.920056 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e80c44-c782-48e8-b12f-441700d27769 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:20.920341 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e80c44-c782-48e8-b12f-441700d27769 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:20.920622 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e80c44-c782-48e8-b12f-441700d27769 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:20.921618 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e80c44-c782-48e8-b12f-441700d27769 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:20.923250 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e80c44-c782-48e8-b12f-441700d27769 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:20.943141 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 962/1953] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:45:20 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:45:20.944724 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e05508-70cc-4c52-bc59-18c11ae03947 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:20.956871 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e05508-70cc-4c52-bc59-18c11ae03947 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:20.961831 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e25686-1850-4570-ac4e-fd5d7de9fccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:20.962451 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e25686-1850-4570-ac4e-fd5d7de9fccd admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:20.962733 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e25686-1850-4570-ac4e-fd5d7de9fccd admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:20.963008 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e25686-1850-4570-ac4e-fd5d7de9fccd admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:20.963848 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e25686-1850-4570-ac4e-fd5d7de9fccd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:20.965476 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e25686-1850-4570-ac4e-fd5d7de9fccd admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:20.976014 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 992/1954] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:45:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:45:20.980624 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e80c44-c782-48e8-b12f-441700d27769 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:20.990955 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e80c44-c782-48e8-b12f-441700d27769 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:20.995564 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9360ef85-99d3-4244-baa0-9e266cd2d15e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:20.996217 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9360ef85-99d3-4244-baa0-9e266cd2d15e admin admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:20.996493 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9360ef85-99d3-4244-baa0-9e266cd2d15e admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:20.996765 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9360ef85-99d3-4244-baa0-9e266cd2d15e admin admin] PATH_INFO: `/v3/projects/1cd889f20c044995b7f74ffcc0f86b4d/users/874dc2e1dee04842b3e1c3895278e6ca/roles/11f8731756fe4caca13efa61b71b11c5` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:21.012599 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9360ef85-99d3-4244-baa0-9e266cd2d15e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1cd889f20c044995b7f74ffcc0f86b4d, user_id=874dc2e1dee04842b3e1c3895278e6ca, role_id=11f8731756fe4caca13efa61b71b11c5)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:21.015957 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9360ef85-99d3-4244-baa0-9e266cd2d15e admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:21.029175 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 963/1955] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:45:20 2022] PUT /identity/v3/projects/1cd889f20c044995b7f74ffcc0f86b4d/users/874dc2e1dee04842b3e1c3895278e6ca/roles/11f8731756fe4caca13efa61b71b11c5 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:45:21.039527 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe23630-45bc-46cb-8c3c-f0296a96d3da None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:21.039823 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe23630-45bc-46cb-8c3c-f0296a96d3da None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:21.040105 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe23630-45bc-46cb-8c3c-f0296a96d3da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:21.068323 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-bfe23630-45bc-46cb-8c3c-f0296a96d3da None None] MFA Rules not processed for user `874dc2e1dee04842b3e1c3895278e6ca`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:45:21.069592 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-bfe23630-45bc-46cb-8c3c-f0296a96d3da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:21.106948 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:45:21.134439 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 993/1956] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:45:21 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:45:21.139051 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9360ef85-99d3-4244-baa0-9e266cd2d15e admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:21.203487 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9360ef85-99d3-4244-baa0-9e266cd2d15e admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:21.209643 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12ea3e9-fe7a-48a0-b271-3ff6360489a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:21.210270 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12ea3e9-fe7a-48a0-b271-3ff6360489a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:21.210545 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12ea3e9-fe7a-48a0-b271-3ff6360489a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:21.210815 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12ea3e9-fe7a-48a0-b271-3ff6360489a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:21.211497 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12ea3e9-fe7a-48a0-b271-3ff6360489a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:21.213401 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12ea3e9-fe7a-48a0-b271-3ff6360489a5 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:21.250291 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 964/1957] 10.222.0.89 () {60 vars in 1153 bytes} [Thu Jul 28 17:45:21 2022] POST /identity/v3/projects => generated 379 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:45:21.254532 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe23630-45bc-46cb-8c3c-f0296a96d3da None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:21.268523 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe23630-45bc-46cb-8c3c-f0296a96d3da None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:21.344598 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b03de74-fb3b-4334-a7c5-d038b3e847b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:21.345716 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b03de74-fb3b-4334-a7c5-d038b3e847b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:21.346016 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b03de74-fb3b-4334-a7c5-d038b3e847b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:21.346295 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b03de74-fb3b-4334-a7c5-d038b3e847b4 admin admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:21.347011 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b03de74-fb3b-4334-a7c5-d038b3e847b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:21.352265 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b03de74-fb3b-4334-a7c5-d038b3e847b4 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:21.373257 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 994/1958] 10.222.0.89 () {60 vars in 1144 bytes} [Thu Jul 28 17:45:21 2022] POST /identity/v3/users => generated 296 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:45:21.377370 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12ea3e9-fe7a-48a0-b271-3ff6360489a5 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:21.388717 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12ea3e9-fe7a-48a0-b271-3ff6360489a5 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:21.394581 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a81c5f-e399-4781-863a-e550e8506b7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:21.395200 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a81c5f-e399-4781-863a-e550e8506b7e admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:21.395484 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a81c5f-e399-4781-863a-e550e8506b7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:21.395760 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a81c5f-e399-4781-863a-e550e8506b7e admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:21.396460 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a81c5f-e399-4781-863a-e550e8506b7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:21.398158 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a81c5f-e399-4781-863a-e550e8506b7e admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:21.410155 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 965/1959] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:45:21 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:45:21.414574 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b03de74-fb3b-4334-a7c5-d038b3e847b4 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:21.425535 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b03de74-fb3b-4334-a7c5-d038b3e847b4 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:21.430258 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb04f35b-65cb-4e99-a2a8-269639d83535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:21.430917 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb04f35b-65cb-4e99-a2a8-269639d83535 admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:21.431188 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb04f35b-65cb-4e99-a2a8-269639d83535 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:21.431466 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb04f35b-65cb-4e99-a2a8-269639d83535 admin admin] PATH_INFO: `/v3/projects/05f93451203f49bd961e59ace9da3a9c/users/2f122bfe50a04573baa0f0d270ab93b0/roles/73e457231a824ea98c28d22610be640c` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:21.448406 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb04f35b-65cb-4e99-a2a8-269639d83535 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05f93451203f49bd961e59ace9da3a9c, user_id=2f122bfe50a04573baa0f0d270ab93b0, role_id=73e457231a824ea98c28d22610be640c)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:21.451100 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb04f35b-65cb-4e99-a2a8-269639d83535 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:21.466396 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 995/1960] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:45:21 2022] PUT /identity/v3/projects/05f93451203f49bd961e59ace9da3a9c/users/2f122bfe50a04573baa0f0d270ab93b0/roles/73e457231a824ea98c28d22610be640c => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 17:45:21.471157 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a81c5f-e399-4781-863a-e550e8506b7e admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:21.481274 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a81c5f-e399-4781-863a-e550e8506b7e admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:21.527661 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:45:21.545716 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ef6de4-685c-4757-b354-c5b6a8f74357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:21.546559 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ef6de4-685c-4757-b354-c5b6a8f74357 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:21.546930 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ef6de4-685c-4757-b354-c5b6a8f74357 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:21.547207 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ef6de4-685c-4757-b354-c5b6a8f74357 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:21.547902 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ef6de4-685c-4757-b354-c5b6a8f74357 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:21.549682 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ef6de4-685c-4757-b354-c5b6a8f74357 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:21.559960 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 966/1961] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:45:21 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:45:21.565081 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb04f35b-65cb-4e99-a2a8-269639d83535 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:21.575687 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb04f35b-65cb-4e99-a2a8-269639d83535 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:21.581779 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2453562e-c2ad-48ad-afdc-de93ff3fcce4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:21.582450 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2453562e-c2ad-48ad-afdc-de93ff3fcce4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:21.582727 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2453562e-c2ad-48ad-afdc-de93ff3fcce4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:21.583002 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2453562e-c2ad-48ad-afdc-de93ff3fcce4 admin admin] PATH_INFO: `/v3/projects/05f93451203f49bd961e59ace9da3a9c/users/2f122bfe50a04573baa0f0d270ab93b0/roles/73e457231a824ea98c28d22610be640c` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:21.585076 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2453562e-c2ad-48ad-afdc-de93ff3fcce4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05f93451203f49bd961e59ace9da3a9c, user_id=2f122bfe50a04573baa0f0d270ab93b0, role_id=73e457231a824ea98c28d22610be640c)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:21.587802 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2453562e-c2ad-48ad-afdc-de93ff3fcce4 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:21.599829 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 996/1962] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:45:21 2022] PUT /identity/v3/projects/05f93451203f49bd961e59ace9da3a9c/users/2f122bfe50a04573baa0f0d270ab93b0/roles/73e457231a824ea98c28d22610be640c => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:45:21.609870 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c34968-f7ac-4fff-aa47-4e09269df201 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:21.610159 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c34968-f7ac-4fff-aa47-4e09269df201 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:21.610437 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c34968-f7ac-4fff-aa47-4e09269df201 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:21.640987 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-c2c34968-f7ac-4fff-aa47-4e09269df201 None None] MFA Rules not processed for user `2f122bfe50a04573baa0f0d270ab93b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:45:21.642174 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-c2c34968-f7ac-4fff-aa47-4e09269df201 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:21.709358 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 967/1963] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:45:21 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:45:21.720692 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2453562e-c2ad-48ad-afdc-de93ff3fcce4 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:21.723139 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-2453562e-c2ad-48ad-afdc-de93ff3fcce4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:21.782670 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2453562e-c2ad-48ad-afdc-de93ff3fcce4 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:21.786425 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20651eb-83b8-42c7-890a-a4658a944140 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96d512d0c84240918dae6c4a3e8b7b79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:21.787034 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20651eb-83b8-42c7-890a-a4658a944140 service cinder] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:21.787313 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20651eb-83b8-42c7-890a-a4658a944140 service cinder] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:21.787614 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20651eb-83b8-42c7-890a-a4658a944140 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:21.797177 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20651eb-83b8-42c7-890a-a4658a944140 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:21.798600 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20651eb-83b8-42c7-890a-a4658a944140 service cinder] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:21.808090 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 997/1964] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:45:21 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:45:21.851806 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c34968-f7ac-4fff-aa47-4e09269df201 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:21.853414 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-c2c34968-f7ac-4fff-aa47-4e09269df201 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:21.865856 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c34968-f7ac-4fff-aa47-4e09269df201 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:21.869932 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ada7f49-c364-479c-a986-e560e2394008 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96d512d0c84240918dae6c4a3e8b7b79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:21.870558 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ada7f49-c364-479c-a986-e560e2394008 service cinder] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:21.870842 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ada7f49-c364-479c-a986-e560e2394008 service cinder] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:21.871117 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ada7f49-c364-479c-a986-e560e2394008 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:21.932918 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ada7f49-c364-479c-a986-e560e2394008 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:21.934397 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ada7f49-c364-479c-a986-e560e2394008 service cinder] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:21.945243 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 968/1965] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:45:21 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:45:23.103403 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20651eb-83b8-42c7-890a-a4658a944140 service cinder] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:23.104865 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-e20651eb-83b8-42c7-890a-a4658a944140 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:23.145851 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:45:23.184280 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20651eb-83b8-42c7-890a-a4658a944140 service cinder] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:23.190959 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab9b6d9-d380-4e6b-ad00-cccb385374ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:23.192033 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab9b6d9-d380-4e6b-ad00-cccb385374ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:23.192477 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab9b6d9-d380-4e6b-ad00-cccb385374ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:23.193069 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab9b6d9-d380-4e6b-ad00-cccb385374ed admin admin] PATH_INFO: `/v3/users/1c8fae03ec514f54ad5e0d14a316bcc8` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:23.194705 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab9b6d9-d380-4e6b-ad00-cccb385374ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c8fae03ec514f54ad5e0d14a316bcc8)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:23.196915 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab9b6d9-d380-4e6b-ad00-cccb385374ed admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:23.259489 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-2ab9b6d9-d380-4e6b-ad00-cccb385374ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c8fae03ec514f54ad5e0d14a316bcc8'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:23.268814 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-2ab9b6d9-d380-4e6b-ad00-cccb385374ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c8fae03ec514f54ad5e0d14a316bcc8'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:23.269845 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-2ab9b6d9-d380-4e6b-ad00-cccb385374ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c8fae03ec514f54ad5e0d14a316bcc8'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:23.279944 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-2ab9b6d9-d380-4e6b-ad00-cccb385374ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c8fae03ec514f54ad5e0d14a316bcc8'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:23.297407 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-2ab9b6d9-d380-4e6b-ad00-cccb385374ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c8fae03ec514f54ad5e0d14a316bcc8'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:23.305289 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 998/1966] 10.222.0.89 () {58 vars in 1224 bytes} [Thu Jul 28 17:45:23 2022] DELETE /identity/v3/users/1c8fae03ec514f54ad5e0d14a316bcc8 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:45:23.309473 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d354c94-fa1c-468d-bae8-4abc46ce9dee None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:23.310324 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d354c94-fa1c-468d-bae8-4abc46ce9dee None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:23.310798 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d354c94-fa1c-468d-bae8-4abc46ce9dee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:23.346291 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-1d354c94-fa1c-468d-bae8-4abc46ce9dee None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:45:23.347949 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-1d354c94-fa1c-468d-bae8-4abc46ce9dee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:23.418726 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 969/1967] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:45:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 17:45:23.430179 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab9b6d9-d380-4e6b-ad00-cccb385374ed admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:23.432207 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-2ab9b6d9-d380-4e6b-ad00-cccb385374ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:23.493016 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab9b6d9-d380-4e6b-ad00-cccb385374ed admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:23.496848 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf77c5a-b327-4d0d-bc26-54792043b895 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76817cd541294331b0023069092e8e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:23.497498 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf77c5a-b327-4d0d-bc26-54792043b895 service neutron] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:23.497779 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf77c5a-b327-4d0d-bc26-54792043b895 service neutron] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:23.498052 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf77c5a-b327-4d0d-bc26-54792043b895 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:23.508092 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf77c5a-b327-4d0d-bc26-54792043b895 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:23.509507 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf77c5a-b327-4d0d-bc26-54792043b895 service neutron] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:23.520101 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 999/1968] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:45:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:45:23.830703 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d354c94-fa1c-468d-bae8-4abc46ce9dee None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:23.830703 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-1d354c94-fa1c-468d-bae8-4abc46ce9dee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:23.845566 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d354c94-fa1c-468d-bae8-4abc46ce9dee None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:23.871902 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f7037a-541f-4311-95b0-050b40d44022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:23.872576 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f7037a-541f-4311-95b0-050b40d44022 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:23.872866 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f7037a-541f-4311-95b0-050b40d44022 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:23.873636 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f7037a-541f-4311-95b0-050b40d44022 admin admin] PATH_INFO: `/v3/projects/e36277abfdd146a4beaa242fc6d0f3cf` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:23.886488 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f7037a-541f-4311-95b0-050b40d44022 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e36277abfdd146a4beaa242fc6d0f3cf)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:23.887547 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f7037a-541f-4311-95b0-050b40d44022 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:23.891358 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:45:23.979072 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-83f7037a-541f-4311-95b0-050b40d44022 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e36277abfdd146a4beaa242fc6d0f3cf', 'request_id': 'req-83f7037a-541f-4311-95b0-050b40d44022'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:23.993372 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-83f7037a-541f-4311-95b0-050b40d44022 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e36277abfdd146a4beaa242fc6d0f3cf', 'request_id': 'req-83f7037a-541f-4311-95b0-050b40d44022'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:24.011007 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-83f7037a-541f-4311-95b0-050b40d44022 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e36277abfdd146a4beaa242fc6d0f3cf', 'request_id': 'req-83f7037a-541f-4311-95b0-050b40d44022'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:24.025714 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-83f7037a-541f-4311-95b0-050b40d44022 admin admin] The token cache is being invalidate because project e36277abfdd146a4beaa242fc6d0f3cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80325) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 17:45:24.026021 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-83f7037a-541f-4311-95b0-050b40d44022 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:24.026861 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-83f7037a-541f-4311-95b0-050b40d44022 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:24.028560 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 970/1969] 10.222.0.89 () {58 vars in 1233 bytes} [Thu Jul 28 17:45:23 2022] DELETE /identity/v3/projects/e36277abfdd146a4beaa242fc6d0f3cf => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:45:24.034419 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee7a62e-d9dc-4539-b43f-1d39ff202d38 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:24.034934 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee7a62e-d9dc-4539-b43f-1d39ff202d38 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:24.035272 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee7a62e-d9dc-4539-b43f-1d39ff202d38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:24.059798 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-2ee7a62e-d9dc-4539-b43f-1d39ff202d38 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:45:24.061204 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-2ee7a62e-d9dc-4539-b43f-1d39ff202d38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:24.122006 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:45:24.133650 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1000/1970] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:45:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 17:45:24.141780 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd898d9d-54cf-4b0f-a97a-bbfb16d5b51e None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:24.142116 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd898d9d-54cf-4b0f-a97a-bbfb16d5b51e None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:24.142400 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd898d9d-54cf-4b0f-a97a-bbfb16d5b51e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:24.162768 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-cd898d9d-54cf-4b0f-a97a-bbfb16d5b51e None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:45:24.163999 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-cd898d9d-54cf-4b0f-a97a-bbfb16d5b51e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:24.173084 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 971/1971] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:45:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:45:24.177516 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee7a62e-d9dc-4539-b43f-1d39ff202d38 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:24.187675 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee7a62e-d9dc-4539-b43f-1d39ff202d38 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:24.192598 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9150ea77-0f95-4b08-850a-a21528833919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:24.193305 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9150ea77-0f95-4b08-850a-a21528833919 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:24.193586 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9150ea77-0f95-4b08-850a-a21528833919 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:24.193977 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9150ea77-0f95-4b08-850a-a21528833919 admin admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:24.194678 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9150ea77-0f95-4b08-850a-a21528833919 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:45:24.195126 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9150ea77-0f95-4b08-850a-a21528833919 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:24.196729 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9150ea77-0f95-4b08-850a-a21528833919 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:24.208825 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1001/1972] 10.222.0.89 () {58 vars in 1153 bytes} [Thu Jul 28 17:45:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:45:24.213553 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd898d9d-54cf-4b0f-a97a-bbfb16d5b51e None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:24.223799 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd898d9d-54cf-4b0f-a97a-bbfb16d5b51e None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:24.228573 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881bd2ef-6703-4a63-9782-dec3b95469af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:24.229212 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881bd2ef-6703-4a63-9782-dec3b95469af admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:24.229497 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881bd2ef-6703-4a63-9782-dec3b95469af admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:24.229806 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881bd2ef-6703-4a63-9782-dec3b95469af admin admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:24.230510 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881bd2ef-6703-4a63-9782-dec3b95469af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:24.232421 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881bd2ef-6703-4a63-9782-dec3b95469af admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:24.264617 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 972/1973] 10.222.0.89 () {60 vars in 1153 bytes} [Thu Jul 28 17:45:24 2022] POST /identity/v3/projects => generated 361 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 17:45:24.269107 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9150ea77-0f95-4b08-850a-a21528833919 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:24.279219 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9150ea77-0f95-4b08-850a-a21528833919 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:24.342623 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda7d870-94d2-4052-8eb8-6311e8e76602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:24.344142 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda7d870-94d2-4052-8eb8-6311e8e76602 admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:24.344630 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda7d870-94d2-4052-8eb8-6311e8e76602 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:24.345104 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda7d870-94d2-4052-8eb8-6311e8e76602 admin admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:24.346345 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda7d870-94d2-4052-8eb8-6311e8e76602 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:24.350069 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda7d870-94d2-4052-8eb8-6311e8e76602 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:24.376673 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1002/1974] 10.222.0.89 () {60 vars in 1144 bytes} [Thu Jul 28 17:45:24 2022] POST /identity/v3/users => generated 281 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:45:24.381375 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881bd2ef-6703-4a63-9782-dec3b95469af admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:24.393630 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881bd2ef-6703-4a63-9782-dec3b95469af admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:24.398999 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a434338a-70f7-4ea0-a6fc-67d10573424d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:24.399952 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a434338a-70f7-4ea0-a6fc-67d10573424d admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:24.400239 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a434338a-70f7-4ea0-a6fc-67d10573424d admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:24.400531 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a434338a-70f7-4ea0-a6fc-67d10573424d admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:24.401244 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a434338a-70f7-4ea0-a6fc-67d10573424d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:24.402837 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a434338a-70f7-4ea0-a6fc-67d10573424d admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:24.414908 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 973/1975] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:45:24 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:45:24.419065 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda7d870-94d2-4052-8eb8-6311e8e76602 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:24.429626 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda7d870-94d2-4052-8eb8-6311e8e76602 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:24.434219 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb83132-aa85-4473-97fa-855fa8197da1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:24.434819 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb83132-aa85-4473-97fa-855fa8197da1 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:24.435096 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb83132-aa85-4473-97fa-855fa8197da1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:24.435364 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb83132-aa85-4473-97fa-855fa8197da1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:24.436042 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb83132-aa85-4473-97fa-855fa8197da1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:24.438032 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb83132-aa85-4473-97fa-855fa8197da1 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:24.448912 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1003/1976] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:45:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:45:24.453986 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a434338a-70f7-4ea0-a6fc-67d10573424d admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:24.463859 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a434338a-70f7-4ea0-a6fc-67d10573424d admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:24.469219 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2b44be-dc12-4e47-a8c5-4f9f1c64de14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:24.469894 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2b44be-dc12-4e47-a8c5-4f9f1c64de14 admin admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:24.470171 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2b44be-dc12-4e47-a8c5-4f9f1c64de14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:24.470440 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2b44be-dc12-4e47-a8c5-4f9f1c64de14 admin admin] PATH_INFO: `/v3/projects/c2e858d02366477ca704e5ac4dbd1f49/users/029cf21a98264a87adef34407ac0824c/roles/11f8731756fe4caca13efa61b71b11c5` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:24.486743 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2b44be-dc12-4e47-a8c5-4f9f1c64de14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2e858d02366477ca704e5ac4dbd1f49, user_id=029cf21a98264a87adef34407ac0824c, role_id=11f8731756fe4caca13efa61b71b11c5)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:24.489417 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2b44be-dc12-4e47-a8c5-4f9f1c64de14 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:24.509257 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 974/1977] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:45:24 2022] PUT /identity/v3/projects/c2e858d02366477ca704e5ac4dbd1f49/users/029cf21a98264a87adef34407ac0824c/roles/11f8731756fe4caca13efa61b71b11c5 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 17:45:24.518717 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56091cb2-d677-4105-989c-e30a6cd7909e None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:24.518985 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56091cb2-d677-4105-989c-e30a6cd7909e None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:24.519266 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56091cb2-d677-4105-989c-e30a6cd7909e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:24.550070 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-56091cb2-d677-4105-989c-e30a6cd7909e None None] MFA Rules not processed for user `029cf21a98264a87adef34407ac0824c`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:45:24.551314 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-56091cb2-d677-4105-989c-e30a6cd7909e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:24.619105 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1004/1978] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:45:24 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:45:24.630905 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2b44be-dc12-4e47-a8c5-4f9f1c64de14 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:24.632592 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-ce2b44be-dc12-4e47-a8c5-4f9f1c64de14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:24.692042 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2b44be-dc12-4e47-a8c5-4f9f1c64de14 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:24.695760 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9935a3b-9284-4519-9a43-8d74f7a3bb2d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96d512d0c84240918dae6c4a3e8b7b79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:24.696399 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9935a3b-9284-4519-9a43-8d74f7a3bb2d service cinder] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:24.696680 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9935a3b-9284-4519-9a43-8d74f7a3bb2d service cinder] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:24.696954 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9935a3b-9284-4519-9a43-8d74f7a3bb2d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:24.706116 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9935a3b-9284-4519-9a43-8d74f7a3bb2d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:24.707545 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9935a3b-9284-4519-9a43-8d74f7a3bb2d service cinder] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:24.716663 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 975/1979] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:45:24 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:45:30.501417 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56091cb2-d677-4105-989c-e30a6cd7909e None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:30.504029 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-56091cb2-d677-4105-989c-e30a6cd7909e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:30.594930 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:45:30.620291 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56091cb2-d677-4105-989c-e30a6cd7909e None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:30.641399 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632fd4a7-6b44-4f7e-b416-923aa2735c48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:30.642517 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632fd4a7-6b44-4f7e-b416-923aa2735c48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:30.643175 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632fd4a7-6b44-4f7e-b416-923aa2735c48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:30.643767 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632fd4a7-6b44-4f7e-b416-923aa2735c48 admin admin] PATH_INFO: `/v3/users/1423e3402c9246edaa0378b048372c1f` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:30.646278 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632fd4a7-6b44-4f7e-b416-923aa2735c48 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1423e3402c9246edaa0378b048372c1f)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:30.648668 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632fd4a7-6b44-4f7e-b416-923aa2735c48 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:30.725340 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-632fd4a7-6b44-4f7e-b416-923aa2735c48 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1423e3402c9246edaa0378b048372c1f'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:30.735938 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-632fd4a7-6b44-4f7e-b416-923aa2735c48 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1423e3402c9246edaa0378b048372c1f'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:30.736858 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-632fd4a7-6b44-4f7e-b416-923aa2735c48 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1423e3402c9246edaa0378b048372c1f'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:30.745354 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-632fd4a7-6b44-4f7e-b416-923aa2735c48 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1423e3402c9246edaa0378b048372c1f'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:30.764650 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-632fd4a7-6b44-4f7e-b416-923aa2735c48 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1423e3402c9246edaa0378b048372c1f'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:30.771717 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1005/1980] 10.222.0.89 () {58 vars in 1224 bytes} [Thu Jul 28 17:45:30 2022] DELETE /identity/v3/users/1423e3402c9246edaa0378b048372c1f => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:45:30.776693 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc93fde-8d74-4a8f-a882-260a0350dcb7 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:30.777243 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc93fde-8d74-4a8f-a882-260a0350dcb7 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:30.777505 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc93fde-8d74-4a8f-a882-260a0350dcb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:30.797470 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-8dc93fde-8d74-4a8f-a882-260a0350dcb7 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:45:30.798778 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-8dc93fde-8d74-4a8f-a882-260a0350dcb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:30.838249 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:45:30.888659 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 976/1981] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:45:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:45:30.899049 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632fd4a7-6b44-4f7e-b416-923aa2735c48 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:30.900811 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-632fd4a7-6b44-4f7e-b416-923aa2735c48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:30.976280 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632fd4a7-6b44-4f7e-b416-923aa2735c48 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:30.981425 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbe2602-27a7-40f8-88e8-3a257f69e591 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76817cd541294331b0023069092e8e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:30.982079 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbe2602-27a7-40f8-88e8-3a257f69e591 service neutron] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:30.982398 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbe2602-27a7-40f8-88e8-3a257f69e591 service neutron] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:30.982632 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbe2602-27a7-40f8-88e8-3a257f69e591 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:30.994940 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbe2602-27a7-40f8-88e8-3a257f69e591 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:30.996514 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbe2602-27a7-40f8-88e8-3a257f69e591 service neutron] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:31.008423 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1006/1982] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:45:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:45:31.410961 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc93fde-8d74-4a8f-a882-260a0350dcb7 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:31.412908 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-8dc93fde-8d74-4a8f-a882-260a0350dcb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:31.427685 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc93fde-8d74-4a8f-a882-260a0350dcb7 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:31.439173 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c1d182-e90e-487a-9d4a-4870584a7e41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:31.439857 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c1d182-e90e-487a-9d4a-4870584a7e41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:31.440193 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c1d182-e90e-487a-9d4a-4870584a7e41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:31.440481 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c1d182-e90e-487a-9d4a-4870584a7e41 admin admin] PATH_INFO: `/v3/projects/15da1db32f2d45a89d2dd022a05b9e4e` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:31.442006 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c1d182-e90e-487a-9d4a-4870584a7e41 admin admin] RBAC: Authorizing `identity:delete_project(project_id=15da1db32f2d45a89d2dd022a05b9e4e)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:31.444126 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c1d182-e90e-487a-9d4a-4870584a7e41 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:31.503258 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-26c1d182-e90e-487a-9d4a-4870584a7e41 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15da1db32f2d45a89d2dd022a05b9e4e', 'request_id': 'req-26c1d182-e90e-487a-9d4a-4870584a7e41'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:31.510920 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-26c1d182-e90e-487a-9d4a-4870584a7e41 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15da1db32f2d45a89d2dd022a05b9e4e', 'request_id': 'req-26c1d182-e90e-487a-9d4a-4870584a7e41'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:31.520777 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-26c1d182-e90e-487a-9d4a-4870584a7e41 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15da1db32f2d45a89d2dd022a05b9e4e', 'request_id': 'req-26c1d182-e90e-487a-9d4a-4870584a7e41'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:31.537776 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-26c1d182-e90e-487a-9d4a-4870584a7e41 admin admin] The token cache is being invalidate because project 15da1db32f2d45a89d2dd022a05b9e4e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80325) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 17:45:31.538081 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-26c1d182-e90e-487a-9d4a-4870584a7e41 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:31.538800 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-26c1d182-e90e-487a-9d4a-4870584a7e41 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:31.540948 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 977/1983] 10.222.0.89 () {58 vars in 1233 bytes} [Thu Jul 28 17:45:31 2022] DELETE /identity/v3/projects/15da1db32f2d45a89d2dd022a05b9e4e => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:45:31.546549 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbe2602-27a7-40f8-88e8-3a257f69e591 service neutron] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:31.558453 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbe2602-27a7-40f8-88e8-3a257f69e591 service neutron] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:31.645767 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a77f82-0952-4b72-97c1-8fd0f231ef25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:31.646450 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a77f82-0952-4b72-97c1-8fd0f231ef25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:31.646902 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a77f82-0952-4b72-97c1-8fd0f231ef25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:31.647214 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a77f82-0952-4b72-97c1-8fd0f231ef25 admin admin] PATH_INFO: `/v3/users/874dc2e1dee04842b3e1c3895278e6ca` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:31.648639 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a77f82-0952-4b72-97c1-8fd0f231ef25 admin admin] RBAC: Authorizing `identity:delete_user(user_id=874dc2e1dee04842b3e1c3895278e6ca)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:31.650805 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a77f82-0952-4b72-97c1-8fd0f231ef25 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:31.667004 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:45:31.711644 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-24a77f82-0952-4b72-97c1-8fd0f231ef25 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '874dc2e1dee04842b3e1c3895278e6ca'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:31.719776 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-24a77f82-0952-4b72-97c1-8fd0f231ef25 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '874dc2e1dee04842b3e1c3895278e6ca'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:31.721046 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-24a77f82-0952-4b72-97c1-8fd0f231ef25 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '874dc2e1dee04842b3e1c3895278e6ca'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:31.731174 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-24a77f82-0952-4b72-97c1-8fd0f231ef25 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '874dc2e1dee04842b3e1c3895278e6ca'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:31.744986 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-24a77f82-0952-4b72-97c1-8fd0f231ef25 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '874dc2e1dee04842b3e1c3895278e6ca'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:31.751155 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1007/1984] 10.222.0.89 () {58 vars in 1224 bytes} [Thu Jul 28 17:45:31 2022] DELETE /identity/v3/users/874dc2e1dee04842b3e1c3895278e6ca => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:45:32.178418 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c1d182-e90e-487a-9d4a-4870584a7e41 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:32.181484 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-26c1d182-e90e-487a-9d4a-4870584a7e41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:32.214650 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c1d182-e90e-487a-9d4a-4870584a7e41 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:32.279968 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26d452e-4937-490f-84ad-e910cf3e659e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:32.281379 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26d452e-4937-490f-84ad-e910cf3e659e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:32.282112 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26d452e-4937-490f-84ad-e910cf3e659e admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:32.282112 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26d452e-4937-490f-84ad-e910cf3e659e admin admin] PATH_INFO: `/v3/projects/1cd889f20c044995b7f74ffcc0f86b4d` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:32.283513 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26d452e-4937-490f-84ad-e910cf3e659e admin admin] RBAC: Authorizing `identity:delete_project(project_id=1cd889f20c044995b7f74ffcc0f86b4d)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:32.286881 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26d452e-4937-490f-84ad-e910cf3e659e admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:32.354255 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-d26d452e-4937-490f-84ad-e910cf3e659e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1cd889f20c044995b7f74ffcc0f86b4d', 'request_id': 'req-d26d452e-4937-490f-84ad-e910cf3e659e'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:32.361744 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-d26d452e-4937-490f-84ad-e910cf3e659e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1cd889f20c044995b7f74ffcc0f86b4d', 'request_id': 'req-d26d452e-4937-490f-84ad-e910cf3e659e'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:32.364122 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:45:32.370178 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-d26d452e-4937-490f-84ad-e910cf3e659e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1cd889f20c044995b7f74ffcc0f86b4d', 'request_id': 'req-d26d452e-4937-490f-84ad-e910cf3e659e'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:32.387348 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-d26d452e-4937-490f-84ad-e910cf3e659e admin admin] The token cache is being invalidate because project 1cd889f20c044995b7f74ffcc0f86b4d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80325) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 17:45:32.387756 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-d26d452e-4937-490f-84ad-e910cf3e659e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:32.388385 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-d26d452e-4937-490f-84ad-e910cf3e659e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:32.396098 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 978/1985] 10.222.0.89 () {58 vars in 1233 bytes} [Thu Jul 28 17:45:32 2022] DELETE /identity/v3/projects/1cd889f20c044995b7f74ffcc0f86b4d => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:45:32.397335 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a77f82-0952-4b72-97c1-8fd0f231ef25 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:32.398845 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-24a77f82-0952-4b72-97c1-8fd0f231ef25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:32.416863 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a77f82-0952-4b72-97c1-8fd0f231ef25 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:32.476701 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5473185-b342-43e6-9fe4-df4efa51355b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:32.477383 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5473185-b342-43e6-9fe4-df4efa51355b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:32.477662 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5473185-b342-43e6-9fe4-df4efa51355b admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:32.477929 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5473185-b342-43e6-9fe4-df4efa51355b admin admin] PATH_INFO: `/v3/users/2f122bfe50a04573baa0f0d270ab93b0` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:32.479102 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5473185-b342-43e6-9fe4-df4efa51355b admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f122bfe50a04573baa0f0d270ab93b0)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:32.481004 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5473185-b342-43e6-9fe4-df4efa51355b admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:32.532064 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-e5473185-b342-43e6-9fe4-df4efa51355b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f122bfe50a04573baa0f0d270ab93b0'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:32.538855 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-e5473185-b342-43e6-9fe4-df4efa51355b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f122bfe50a04573baa0f0d270ab93b0'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:32.539542 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-e5473185-b342-43e6-9fe4-df4efa51355b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f122bfe50a04573baa0f0d270ab93b0'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:32.546881 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-e5473185-b342-43e6-9fe4-df4efa51355b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f122bfe50a04573baa0f0d270ab93b0'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:32.558586 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-e5473185-b342-43e6-9fe4-df4efa51355b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f122bfe50a04573baa0f0d270ab93b0'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:32.565240 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1008/1986] 10.222.0.89 () {58 vars in 1224 bytes} [Thu Jul 28 17:45:32 2022] DELETE /identity/v3/users/2f122bfe50a04573baa0f0d270ab93b0 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:45:32.939818 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26d452e-4937-490f-84ad-e910cf3e659e admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:32.941755 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-d26d452e-4937-490f-84ad-e910cf3e659e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:32.966382 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26d452e-4937-490f-84ad-e910cf3e659e admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:33.030610 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e53149-4987-40ea-b93e-5af4fa8cdd3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:33.031373 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e53149-4987-40ea-b93e-5af4fa8cdd3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:33.031650 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e53149-4987-40ea-b93e-5af4fa8cdd3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:33.031925 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e53149-4987-40ea-b93e-5af4fa8cdd3b admin admin] PATH_INFO: `/v3/projects/05f93451203f49bd961e59ace9da3a9c` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:33.033200 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e53149-4987-40ea-b93e-5af4fa8cdd3b admin admin] RBAC: Authorizing `identity:delete_project(project_id=05f93451203f49bd961e59ace9da3a9c)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:33.035156 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e53149-4987-40ea-b93e-5af4fa8cdd3b admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:33.085743 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-d0e53149-4987-40ea-b93e-5af4fa8cdd3b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05f93451203f49bd961e59ace9da3a9c', 'request_id': 'req-d0e53149-4987-40ea-b93e-5af4fa8cdd3b'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:33.093971 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-d0e53149-4987-40ea-b93e-5af4fa8cdd3b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05f93451203f49bd961e59ace9da3a9c', 'request_id': 'req-d0e53149-4987-40ea-b93e-5af4fa8cdd3b'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:33.106144 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-d0e53149-4987-40ea-b93e-5af4fa8cdd3b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05f93451203f49bd961e59ace9da3a9c', 'request_id': 'req-d0e53149-4987-40ea-b93e-5af4fa8cdd3b'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:33.120236 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-d0e53149-4987-40ea-b93e-5af4fa8cdd3b admin admin] The token cache is being invalidate because project 05f93451203f49bd961e59ace9da3a9c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80325) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 17:45:33.120624 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-d0e53149-4987-40ea-b93e-5af4fa8cdd3b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:33.121442 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-d0e53149-4987-40ea-b93e-5af4fa8cdd3b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:33.122992 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 979/1987] 10.222.0.89 () {58 vars in 1233 bytes} [Thu Jul 28 17:45:32 2022] DELETE /identity/v3/projects/05f93451203f49bd961e59ace9da3a9c => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:45:33.130038 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be24b68c-4829-4466-a3f0-e5b45eac0e2f None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:33.130341 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be24b68c-4829-4466-a3f0-e5b45eac0e2f None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:33.130733 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be24b68c-4829-4466-a3f0-e5b45eac0e2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:33.152107 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-be24b68c-4829-4466-a3f0-e5b45eac0e2f None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:45:33.154035 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-be24b68c-4829-4466-a3f0-e5b45eac0e2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:33.161482 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:45:33.219890 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1009/1988] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:45:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 17:45:33.231679 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdd32a6-df40-4bfe-b92c-7819b97e74f6 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:33.232134 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdd32a6-df40-4bfe-b92c-7819b97e74f6 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:33.232222 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdd32a6-df40-4bfe-b92c-7819b97e74f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:33.259681 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-8bdd32a6-df40-4bfe-b92c-7819b97e74f6 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:45:33.261440 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-8bdd32a6-df40-4bfe-b92c-7819b97e74f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:33.275675 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 980/1989] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:45:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:45:33.276646 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be24b68c-4829-4466-a3f0-e5b45eac0e2f None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:33.293407 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be24b68c-4829-4466-a3f0-e5b45eac0e2f None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:33.302842 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3752318-9871-4144-ae7a-98ee8bd0b512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:33.303792 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3752318-9871-4144-ae7a-98ee8bd0b512 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:33.304173 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3752318-9871-4144-ae7a-98ee8bd0b512 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:33.304545 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3752318-9871-4144-ae7a-98ee8bd0b512 admin admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:33.305959 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3752318-9871-4144-ae7a-98ee8bd0b512 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:45:33.306374 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3752318-9871-4144-ae7a-98ee8bd0b512 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:33.308109 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3752318-9871-4144-ae7a-98ee8bd0b512 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:33.322983 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1010/1990] 10.222.0.89 () {58 vars in 1153 bytes} [Thu Jul 28 17:45:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:45:33.328555 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdd32a6-df40-4bfe-b92c-7819b97e74f6 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:33.338772 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdd32a6-df40-4bfe-b92c-7819b97e74f6 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:33.347637 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a27097-2b14-485d-a837-7f279c3967ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:33.348315 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a27097-2b14-485d-a837-7f279c3967ab admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:33.348527 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a27097-2b14-485d-a837-7f279c3967ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:33.348800 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a27097-2b14-485d-a837-7f279c3967ab admin admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:33.349513 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a27097-2b14-485d-a837-7f279c3967ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:33.351414 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a27097-2b14-485d-a837-7f279c3967ab admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:33.387221 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 981/1991] 10.222.0.89 () {60 vars in 1153 bytes} [Thu Jul 28 17:45:33 2022] POST /identity/v3/projects => generated 387 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:45:33.388534 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3752318-9871-4144-ae7a-98ee8bd0b512 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:33.400261 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3752318-9871-4144-ae7a-98ee8bd0b512 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:33.458907 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea7b2ad-a2d2-40f1-8a31-17b550b0812c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:33.459553 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea7b2ad-a2d2-40f1-8a31-17b550b0812c admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:33.459835 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea7b2ad-a2d2-40f1-8a31-17b550b0812c admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:33.460106 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea7b2ad-a2d2-40f1-8a31-17b550b0812c admin admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:33.460767 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea7b2ad-a2d2-40f1-8a31-17b550b0812c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:33.462742 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea7b2ad-a2d2-40f1-8a31-17b550b0812c admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:33.485943 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1011/1992] 10.222.0.89 () {60 vars in 1144 bytes} [Thu Jul 28 17:45:33 2022] POST /identity/v3/users => generated 294 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:45:33.490471 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a27097-2b14-485d-a837-7f279c3967ab admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:33.501335 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a27097-2b14-485d-a837-7f279c3967ab admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:33.506004 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0fc984-e18d-41ea-9f63-29e86ef1dc75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:33.506641 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0fc984-e18d-41ea-9f63-29e86ef1dc75 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:33.506921 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0fc984-e18d-41ea-9f63-29e86ef1dc75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:33.507192 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0fc984-e18d-41ea-9f63-29e86ef1dc75 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:33.507879 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0fc984-e18d-41ea-9f63-29e86ef1dc75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:33.509485 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0fc984-e18d-41ea-9f63-29e86ef1dc75 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:33.524886 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 982/1993] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:45:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:45:33.526339 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea7b2ad-a2d2-40f1-8a31-17b550b0812c admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:33.538424 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea7b2ad-a2d2-40f1-8a31-17b550b0812c admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:33.544989 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ed52a6-add4-48da-90a5-dbb25b2f4187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:33.545811 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ed52a6-add4-48da-90a5-dbb25b2f4187 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:33.546409 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ed52a6-add4-48da-90a5-dbb25b2f4187 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:33.546805 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ed52a6-add4-48da-90a5-dbb25b2f4187 admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:33.547989 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ed52a6-add4-48da-90a5-dbb25b2f4187 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:33.549366 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ed52a6-add4-48da-90a5-dbb25b2f4187 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:33.560684 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1012/1994] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:45:33 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:45:33.564961 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0fc984-e18d-41ea-9f63-29e86ef1dc75 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:33.575164 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0fc984-e18d-41ea-9f63-29e86ef1dc75 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:33.581701 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4ed895-2b7a-47a2-a296-64cca1938c2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:33.582647 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4ed895-2b7a-47a2-a296-64cca1938c2e admin admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:33.583152 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4ed895-2b7a-47a2-a296-64cca1938c2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:33.583424 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4ed895-2b7a-47a2-a296-64cca1938c2e admin admin] PATH_INFO: `/v3/projects/83b48d04699c46579d7a813f21efad9d/users/ea59c114b697482c9de1bfcd8cdec8a3/roles/11f8731756fe4caca13efa61b71b11c5` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:33.600777 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4ed895-2b7a-47a2-a296-64cca1938c2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=83b48d04699c46579d7a813f21efad9d, user_id=ea59c114b697482c9de1bfcd8cdec8a3, role_id=11f8731756fe4caca13efa61b71b11c5)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:33.603647 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4ed895-2b7a-47a2-a296-64cca1938c2e admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:33.619888 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 983/1995] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:45:33 2022] PUT /identity/v3/projects/83b48d04699c46579d7a813f21efad9d/users/ea59c114b697482c9de1bfcd8cdec8a3/roles/11f8731756fe4caca13efa61b71b11c5 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:45:33.631329 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b998ea-7c0c-4ea7-82c3-dfe145f6d4b2 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:33.631626 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b998ea-7c0c-4ea7-82c3-dfe145f6d4b2 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:33.631905 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b998ea-7c0c-4ea7-82c3-dfe145f6d4b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:33.661027 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-16b998ea-7c0c-4ea7-82c3-dfe145f6d4b2 None None] MFA Rules not processed for user `ea59c114b697482c9de1bfcd8cdec8a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:45:33.662231 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-16b998ea-7c0c-4ea7-82c3-dfe145f6d4b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:33.728316 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1013/1996] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:45:33 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:45:33.732999 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4ed895-2b7a-47a2-a296-64cca1938c2e admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:33.745757 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:45:33.796517 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4ed895-2b7a-47a2-a296-64cca1938c2e admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:33.801129 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435e384f-b1d2-48fd-bb41-a0dd6de13b4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:33.801770 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435e384f-b1d2-48fd-bb41-a0dd6de13b4a admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:33.802052 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435e384f-b1d2-48fd-bb41-a0dd6de13b4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:33.802356 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435e384f-b1d2-48fd-bb41-a0dd6de13b4a admin admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:33.803025 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435e384f-b1d2-48fd-bb41-a0dd6de13b4a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:33.804930 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435e384f-b1d2-48fd-bb41-a0dd6de13b4a admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:33.837290 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 984/1997] 10.222.0.89 () {60 vars in 1153 bytes} [Thu Jul 28 17:45:33 2022] POST /identity/v3/projects => generated 387 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 17:45:33.841560 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b998ea-7c0c-4ea7-82c3-dfe145f6d4b2 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:33.852921 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b998ea-7c0c-4ea7-82c3-dfe145f6d4b2 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:33.856675 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:45:33.918983 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cb4a8c-7ca7-4517-98db-19d695e1b486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:33.919608 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cb4a8c-7ca7-4517-98db-19d695e1b486 admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:33.919896 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cb4a8c-7ca7-4517-98db-19d695e1b486 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:33.920202 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cb4a8c-7ca7-4517-98db-19d695e1b486 admin admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:33.921019 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cb4a8c-7ca7-4517-98db-19d695e1b486 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:33.922973 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cb4a8c-7ca7-4517-98db-19d695e1b486 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:33.944937 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1014/1998] 10.222.0.89 () {60 vars in 1144 bytes} [Thu Jul 28 17:45:33 2022] POST /identity/v3/users => generated 300 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:45:33.948956 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435e384f-b1d2-48fd-bb41-a0dd6de13b4a admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:33.960030 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435e384f-b1d2-48fd-bb41-a0dd6de13b4a admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:33.964711 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b04714-2efe-47f2-9170-d1d0acc06ba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:33.965366 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b04714-2efe-47f2-9170-d1d0acc06ba8 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:33.965646 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b04714-2efe-47f2-9170-d1d0acc06ba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:33.965918 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b04714-2efe-47f2-9170-d1d0acc06ba8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:33.966616 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b04714-2efe-47f2-9170-d1d0acc06ba8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:33.968361 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b04714-2efe-47f2-9170-d1d0acc06ba8 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:33.979232 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 985/1999] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:45:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:45:33.983139 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cb4a8c-7ca7-4517-98db-19d695e1b486 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:33.993667 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cb4a8c-7ca7-4517-98db-19d695e1b486 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:33.998583 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d762d62d-1c5f-4b0c-a2ad-823593915460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:34.000554 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d762d62d-1c5f-4b0c-a2ad-823593915460 admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:34.000837 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d762d62d-1c5f-4b0c-a2ad-823593915460 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:34.001117 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d762d62d-1c5f-4b0c-a2ad-823593915460 admin admin] PATH_INFO: `/v3/projects/bf2431b1370240bc9052a948800c3829/users/fa346a3bdd5d40f5a7442d8b2542b66e/roles/73e457231a824ea98c28d22610be640c` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:34.017967 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d762d62d-1c5f-4b0c-a2ad-823593915460 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf2431b1370240bc9052a948800c3829, user_id=fa346a3bdd5d40f5a7442d8b2542b66e, role_id=73e457231a824ea98c28d22610be640c)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:34.020656 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d762d62d-1c5f-4b0c-a2ad-823593915460 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:34.034624 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1015/2000] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:45:33 2022] PUT /identity/v3/projects/bf2431b1370240bc9052a948800c3829/users/fa346a3bdd5d40f5a7442d8b2542b66e/roles/73e457231a824ea98c28d22610be640c => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 17:45:34.038636 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b04714-2efe-47f2-9170-d1d0acc06ba8 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:34.049678 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b04714-2efe-47f2-9170-d1d0acc06ba8 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:34.109388 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00e7d12-8e46-42dd-b77a-861e397f0b58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:34.110024 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00e7d12-8e46-42dd-b77a-861e397f0b58 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:34.110303 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00e7d12-8e46-42dd-b77a-861e397f0b58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:34.110573 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00e7d12-8e46-42dd-b77a-861e397f0b58 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:34.111269 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00e7d12-8e46-42dd-b77a-861e397f0b58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:34.112890 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00e7d12-8e46-42dd-b77a-861e397f0b58 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:34.127491 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 986/2001] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:45:34 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:45:34.131477 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d762d62d-1c5f-4b0c-a2ad-823593915460 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:34.142339 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d762d62d-1c5f-4b0c-a2ad-823593915460 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:34.147394 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1b7801-1a06-4266-b109-816373896e91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:34.148057 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1b7801-1a06-4266-b109-816373896e91 admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:34.148336 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1b7801-1a06-4266-b109-816373896e91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:34.148610 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1b7801-1a06-4266-b109-816373896e91 admin admin] PATH_INFO: `/v3/projects/bf2431b1370240bc9052a948800c3829/users/fa346a3bdd5d40f5a7442d8b2542b66e/roles/73e457231a824ea98c28d22610be640c` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:34.150693 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1b7801-1a06-4266-b109-816373896e91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf2431b1370240bc9052a948800c3829, user_id=fa346a3bdd5d40f5a7442d8b2542b66e, role_id=73e457231a824ea98c28d22610be640c)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:34.153535 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1b7801-1a06-4266-b109-816373896e91 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:34.164575 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1016/2002] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:45:34 2022] PUT /identity/v3/projects/bf2431b1370240bc9052a948800c3829/users/fa346a3bdd5d40f5a7442d8b2542b66e/roles/73e457231a824ea98c28d22610be640c => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:45:34.176279 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91931475-d7b9-4057-b153-29bb314c34a2 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:34.176579 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91931475-d7b9-4057-b153-29bb314c34a2 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:34.176963 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91931475-d7b9-4057-b153-29bb314c34a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:34.206725 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-91931475-d7b9-4057-b153-29bb314c34a2 None None] MFA Rules not processed for user `fa346a3bdd5d40f5a7442d8b2542b66e`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:45:34.208097 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-91931475-d7b9-4057-b153-29bb314c34a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:34.279330 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 987/2003] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:45:34 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 17:45:34.290809 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1b7801-1a06-4266-b109-816373896e91 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:34.292668 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-3b1b7801-1a06-4266-b109-816373896e91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:34.362515 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1b7801-1a06-4266-b109-816373896e91 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:34.366285 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba741cc9-a240-4dde-bc0b-c53460d2d5d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96d512d0c84240918dae6c4a3e8b7b79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:34.366901 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba741cc9-a240-4dde-bc0b-c53460d2d5d3 service cinder] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:34.367184 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba741cc9-a240-4dde-bc0b-c53460d2d5d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:34.367455 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba741cc9-a240-4dde-bc0b-c53460d2d5d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:34.377299 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba741cc9-a240-4dde-bc0b-c53460d2d5d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:34.378722 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba741cc9-a240-4dde-bc0b-c53460d2d5d3 service cinder] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:34.388680 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1017/2004] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:45:34 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:45:34.997788 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91931475-d7b9-4057-b153-29bb314c34a2 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:35.007089 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:45:35.062788 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91931475-d7b9-4057-b153-29bb314c34a2 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:35.067470 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769332cd-479d-470c-88f7-2d0b49e371ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:35.068113 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769332cd-479d-470c-88f7-2d0b49e371ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:35.068396 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769332cd-479d-470c-88f7-2d0b49e371ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:35.068674 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769332cd-479d-470c-88f7-2d0b49e371ef admin admin] PATH_INFO: `/v3/users/ea59c114b697482c9de1bfcd8cdec8a3` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:35.069874 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769332cd-479d-470c-88f7-2d0b49e371ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea59c114b697482c9de1bfcd8cdec8a3)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:35.072731 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769332cd-479d-470c-88f7-2d0b49e371ef admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:35.123901 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-769332cd-479d-470c-88f7-2d0b49e371ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea59c114b697482c9de1bfcd8cdec8a3'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:35.131778 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-769332cd-479d-470c-88f7-2d0b49e371ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea59c114b697482c9de1bfcd8cdec8a3'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:35.141072 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-769332cd-479d-470c-88f7-2d0b49e371ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea59c114b697482c9de1bfcd8cdec8a3'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:35.141972 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-769332cd-479d-470c-88f7-2d0b49e371ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea59c114b697482c9de1bfcd8cdec8a3'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:35.142588 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-769332cd-479d-470c-88f7-2d0b49e371ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea59c114b697482c9de1bfcd8cdec8a3'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:35.161369 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 988/2005] 10.222.0.89 () {58 vars in 1224 bytes} [Thu Jul 28 17:45:34 2022] DELETE /identity/v3/users/ea59c114b697482c9de1bfcd8cdec8a3 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:45:35.166165 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e73122a-18db-4924-a5af-dcfd5dcfcb29 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:35.166165 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e73122a-18db-4924-a5af-dcfd5dcfcb29 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:35.166804 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e73122a-18db-4924-a5af-dcfd5dcfcb29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:35.185697 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-4e73122a-18db-4924-a5af-dcfd5dcfcb29 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:45:35.187555 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-4e73122a-18db-4924-a5af-dcfd5dcfcb29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:35.249371 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1018/2006] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:45:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:45:35.260402 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769332cd-479d-470c-88f7-2d0b49e371ef admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:35.261839 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-769332cd-479d-470c-88f7-2d0b49e371ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:35.319419 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769332cd-479d-470c-88f7-2d0b49e371ef admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:35.323280 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2a56de-01a9-4cbf-bfeb-c033e2c65a11 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76817cd541294331b0023069092e8e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:35.323936 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2a56de-01a9-4cbf-bfeb-c033e2c65a11 service neutron] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:35.324243 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2a56de-01a9-4cbf-bfeb-c033e2c65a11 service neutron] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:35.324520 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2a56de-01a9-4cbf-bfeb-c033e2c65a11 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:35.333934 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2a56de-01a9-4cbf-bfeb-c033e2c65a11 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:35.335363 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2a56de-01a9-4cbf-bfeb-c033e2c65a11 service neutron] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:35.345932 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 989/2007] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:45:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:45:35.703364 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e73122a-18db-4924-a5af-dcfd5dcfcb29 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:35.704855 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-4e73122a-18db-4924-a5af-dcfd5dcfcb29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:35.712065 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:45:35.723804 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e73122a-18db-4924-a5af-dcfd5dcfcb29 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:35.728743 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee171cc-8fbf-40e6-bbbb-42368e0363e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:35.729449 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee171cc-8fbf-40e6-bbbb-42368e0363e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:35.729724 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee171cc-8fbf-40e6-bbbb-42368e0363e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:35.730060 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee171cc-8fbf-40e6-bbbb-42368e0363e4 admin admin] PATH_INFO: `/v3/projects/83b48d04699c46579d7a813f21efad9d` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:35.731216 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee171cc-8fbf-40e6-bbbb-42368e0363e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=83b48d04699c46579d7a813f21efad9d)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:35.733074 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee171cc-8fbf-40e6-bbbb-42368e0363e4 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:35.786335 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-8ee171cc-8fbf-40e6-bbbb-42368e0363e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '83b48d04699c46579d7a813f21efad9d', 'request_id': 'req-8ee171cc-8fbf-40e6-bbbb-42368e0363e4'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:35.795632 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-8ee171cc-8fbf-40e6-bbbb-42368e0363e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '83b48d04699c46579d7a813f21efad9d', 'request_id': 'req-8ee171cc-8fbf-40e6-bbbb-42368e0363e4'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:35.802528 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-8ee171cc-8fbf-40e6-bbbb-42368e0363e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '83b48d04699c46579d7a813f21efad9d', 'request_id': 'req-8ee171cc-8fbf-40e6-bbbb-42368e0363e4'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:35.820112 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-8ee171cc-8fbf-40e6-bbbb-42368e0363e4 admin admin] The token cache is being invalidate because project 83b48d04699c46579d7a813f21efad9d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80324) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 17:45:35.820403 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-8ee171cc-8fbf-40e6-bbbb-42368e0363e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:35.821050 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-8ee171cc-8fbf-40e6-bbbb-42368e0363e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:35.822622 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1019/2008] 10.222.0.89 () {58 vars in 1233 bytes} [Thu Jul 28 17:45:35 2022] DELETE /identity/v3/projects/83b48d04699c46579d7a813f21efad9d => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 17:45:35.826623 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2a56de-01a9-4cbf-bfeb-c033e2c65a11 service neutron] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:35.837488 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2a56de-01a9-4cbf-bfeb-c033e2c65a11 service neutron] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:35.864937 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:45:35.906918 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa933b2-de48-42d6-b9cc-b3dbfc17311a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:35.907688 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa933b2-de48-42d6-b9cc-b3dbfc17311a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:35.908195 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa933b2-de48-42d6-b9cc-b3dbfc17311a admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:35.908535 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa933b2-de48-42d6-b9cc-b3dbfc17311a admin admin] PATH_INFO: `/v3/users/fa346a3bdd5d40f5a7442d8b2542b66e` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:35.910033 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa933b2-de48-42d6-b9cc-b3dbfc17311a admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa346a3bdd5d40f5a7442d8b2542b66e)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:35.912169 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa933b2-de48-42d6-b9cc-b3dbfc17311a admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:35.973485 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-9aa933b2-de48-42d6-b9cc-b3dbfc17311a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa346a3bdd5d40f5a7442d8b2542b66e'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:35.982124 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-9aa933b2-de48-42d6-b9cc-b3dbfc17311a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa346a3bdd5d40f5a7442d8b2542b66e'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:35.990393 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-9aa933b2-de48-42d6-b9cc-b3dbfc17311a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa346a3bdd5d40f5a7442d8b2542b66e'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:35.991177 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-9aa933b2-de48-42d6-b9cc-b3dbfc17311a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa346a3bdd5d40f5a7442d8b2542b66e'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:35.992053 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-9aa933b2-de48-42d6-b9cc-b3dbfc17311a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa346a3bdd5d40f5a7442d8b2542b66e'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:36.012921 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 990/2009] 10.222.0.89 () {58 vars in 1224 bytes} [Thu Jul 28 17:45:35 2022] DELETE /identity/v3/users/fa346a3bdd5d40f5a7442d8b2542b66e => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:45:36.289415 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee171cc-8fbf-40e6-bbbb-42368e0363e4 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:36.290904 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-8ee171cc-8fbf-40e6-bbbb-42368e0363e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:36.322600 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee171cc-8fbf-40e6-bbbb-42368e0363e4 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:36.387428 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768723ea-7e59-411f-b38f-ce37266e218b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:36.388080 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768723ea-7e59-411f-b38f-ce37266e218b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:36.388360 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768723ea-7e59-411f-b38f-ce37266e218b admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:36.388633 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768723ea-7e59-411f-b38f-ce37266e218b admin admin] PATH_INFO: `/v3/projects/bf2431b1370240bc9052a948800c3829` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:36.390087 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768723ea-7e59-411f-b38f-ce37266e218b admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf2431b1370240bc9052a948800c3829)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:36.392307 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768723ea-7e59-411f-b38f-ce37266e218b admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:36.444071 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-768723ea-7e59-411f-b38f-ce37266e218b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf2431b1370240bc9052a948800c3829', 'request_id': 'req-768723ea-7e59-411f-b38f-ce37266e218b'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:36.449483 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-768723ea-7e59-411f-b38f-ce37266e218b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf2431b1370240bc9052a948800c3829', 'request_id': 'req-768723ea-7e59-411f-b38f-ce37266e218b'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:36.456579 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-768723ea-7e59-411f-b38f-ce37266e218b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf2431b1370240bc9052a948800c3829', 'request_id': 'req-768723ea-7e59-411f-b38f-ce37266e218b'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:36.475916 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-768723ea-7e59-411f-b38f-ce37266e218b admin admin] The token cache is being invalidate because project bf2431b1370240bc9052a948800c3829 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80324) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 17:45:36.476210 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-768723ea-7e59-411f-b38f-ce37266e218b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:36.476869 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-768723ea-7e59-411f-b38f-ce37266e218b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:36.478419 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1020/2010] 10.222.0.89 () {58 vars in 1233 bytes} [Thu Jul 28 17:45:36 2022] DELETE /identity/v3/projects/bf2431b1370240bc9052a948800c3829 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:45:36.483798 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c486ca2-11a0-43ad-9720-7b2482f0a1de None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:36.484290 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c486ca2-11a0-43ad-9720-7b2482f0a1de None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:36.484406 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c486ca2-11a0-43ad-9720-7b2482f0a1de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:36.506036 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-4c486ca2-11a0-43ad-9720-7b2482f0a1de None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:45:36.507253 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-4c486ca2-11a0-43ad-9720-7b2482f0a1de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:36.574059 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 991/2011] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:45:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 17:45:36.582261 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a759c857-08c2-49b3-a35f-8796bfd8a33b None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:36.582571 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a759c857-08c2-49b3-a35f-8796bfd8a33b None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:36.582835 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a759c857-08c2-49b3-a35f-8796bfd8a33b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:36.605307 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-a759c857-08c2-49b3-a35f-8796bfd8a33b None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:45:36.606510 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-a759c857-08c2-49b3-a35f-8796bfd8a33b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:36.614512 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1021/2012] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:45:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:45:36.618819 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c486ca2-11a0-43ad-9720-7b2482f0a1de None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:36.629942 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c486ca2-11a0-43ad-9720-7b2482f0a1de None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:36.635381 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf01161-a9dc-4930-935b-84a707cac974 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:36.636025 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf01161-a9dc-4930-935b-84a707cac974 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:36.636305 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf01161-a9dc-4930-935b-84a707cac974 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:36.636574 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf01161-a9dc-4930-935b-84a707cac974 admin admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:36.637315 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf01161-a9dc-4930-935b-84a707cac974 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:45:36.637608 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf01161-a9dc-4930-935b-84a707cac974 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:36.639184 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf01161-a9dc-4930-935b-84a707cac974 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:36.652107 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 992/2013] 10.222.0.89 () {58 vars in 1153 bytes} [Thu Jul 28 17:45:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:45:36.658373 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a759c857-08c2-49b3-a35f-8796bfd8a33b None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:36.667875 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a759c857-08c2-49b3-a35f-8796bfd8a33b None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:36.673609 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbac71ab-a537-453b-8204-e4b2362e86c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:36.674375 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbac71ab-a537-453b-8204-e4b2362e86c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:36.674754 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbac71ab-a537-453b-8204-e4b2362e86c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:36.675128 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbac71ab-a537-453b-8204-e4b2362e86c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:36.675910 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbac71ab-a537-453b-8204-e4b2362e86c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:36.678065 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbac71ab-a537-453b-8204-e4b2362e86c4 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:36.684357 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:45:36.713746 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1022/2014] 10.222.0.89 () {60 vars in 1153 bytes} [Thu Jul 28 17:45:36 2022] POST /identity/v3/projects => generated 371 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 17:45:36.716850 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf01161-a9dc-4930-935b-84a707cac974 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:36.728089 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf01161-a9dc-4930-935b-84a707cac974 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:36.771864 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:45:36.802611 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac26177c-1739-4b20-b0a3-e6c2a1b58a10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:36.804751 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac26177c-1739-4b20-b0a3-e6c2a1b58a10 admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:36.805038 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac26177c-1739-4b20-b0a3-e6c2a1b58a10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:36.805343 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac26177c-1739-4b20-b0a3-e6c2a1b58a10 admin admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:36.806039 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac26177c-1739-4b20-b0a3-e6c2a1b58a10 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:36.807992 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac26177c-1739-4b20-b0a3-e6c2a1b58a10 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:36.835128 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 993/2015] 10.222.0.89 () {60 vars in 1144 bytes} [Thu Jul 28 17:45:36 2022] POST /identity/v3/users => generated 286 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:45:36.839253 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbac71ab-a537-453b-8204-e4b2362e86c4 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:36.850185 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbac71ab-a537-453b-8204-e4b2362e86c4 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:36.854837 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60f76fe-61d4-4d4e-8ea7-9c504cc5efe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:36.855433 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60f76fe-61d4-4d4e-8ea7-9c504cc5efe2 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:36.855712 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60f76fe-61d4-4d4e-8ea7-9c504cc5efe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:36.855983 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60f76fe-61d4-4d4e-8ea7-9c504cc5efe2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:36.856672 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60f76fe-61d4-4d4e-8ea7-9c504cc5efe2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:36.858384 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60f76fe-61d4-4d4e-8ea7-9c504cc5efe2 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:36.870062 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1023/2016] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:45:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:45:36.875167 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac26177c-1739-4b20-b0a3-e6c2a1b58a10 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:36.886181 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac26177c-1739-4b20-b0a3-e6c2a1b58a10 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:36.891771 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1074a8c5-b128-40de-8542-aa730eb0b27a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:36.892407 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1074a8c5-b128-40de-8542-aa730eb0b27a admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:36.892794 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1074a8c5-b128-40de-8542-aa730eb0b27a admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:36.893306 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1074a8c5-b128-40de-8542-aa730eb0b27a admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:36.894003 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1074a8c5-b128-40de-8542-aa730eb0b27a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:36.895715 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1074a8c5-b128-40de-8542-aa730eb0b27a admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:36.907626 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 994/2017] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:45:36 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:45:36.911794 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60f76fe-61d4-4d4e-8ea7-9c504cc5efe2 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:36.922415 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60f76fe-61d4-4d4e-8ea7-9c504cc5efe2 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:36.927251 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e16845-7a25-4a70-aad4-f71c531574fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:36.927953 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e16845-7a25-4a70-aad4-f71c531574fa admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:36.928225 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e16845-7a25-4a70-aad4-f71c531574fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:36.942757 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e16845-7a25-4a70-aad4-f71c531574fa admin admin] PATH_INFO: `/v3/projects/6bd0a6d50b6345339f0e6adccd157202/users/82ea018632fb4487b081f23fca5b15a3/roles/11f8731756fe4caca13efa61b71b11c5` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:36.971374 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e16845-7a25-4a70-aad4-f71c531574fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=6bd0a6d50b6345339f0e6adccd157202, user_id=82ea018632fb4487b081f23fca5b15a3, role_id=11f8731756fe4caca13efa61b71b11c5)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:36.974497 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e16845-7a25-4a70-aad4-f71c531574fa admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:36.998675 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1024/2018] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:45:36 2022] PUT /identity/v3/projects/6bd0a6d50b6345339f0e6adccd157202/users/82ea018632fb4487b081f23fca5b15a3/roles/11f8731756fe4caca13efa61b71b11c5 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:45:37.004729 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ca16ab-1614-4617-8b58-1ba53789987f None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:37.005032 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ca16ab-1614-4617-8b58-1ba53789987f None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:37.005323 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ca16ab-1614-4617-8b58-1ba53789987f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:37.033400 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-c7ca16ab-1614-4617-8b58-1ba53789987f None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:45:37.034851 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-c7ca16ab-1614-4617-8b58-1ba53789987f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:37.112315 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 995/2019] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:45:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:45:37.123815 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e16845-7a25-4a70-aad4-f71c531574fa admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:37.126335 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-00e16845-7a25-4a70-aad4-f71c531574fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:37.222296 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e16845-7a25-4a70-aad4-f71c531574fa admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:37.226570 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d5f9a9-2bd0-4697-8a6f-d6a90c0715a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76817cd541294331b0023069092e8e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:37.227341 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d5f9a9-2bd0-4697-8a6f-d6a90c0715a3 service neutron] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:37.227626 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d5f9a9-2bd0-4697-8a6f-d6a90c0715a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:37.228192 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d5f9a9-2bd0-4697-8a6f-d6a90c0715a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:37.242006 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d5f9a9-2bd0-4697-8a6f-d6a90c0715a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:37.243784 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d5f9a9-2bd0-4697-8a6f-d6a90c0715a3 service neutron] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:37.267493 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1025/2020] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:45:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:45:38.406515 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dec5994-83a7-42be-bfaf-78c51d4bbd82 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:38.407968 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dec5994-83a7-42be-bfaf-78c51d4bbd82 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:38.407968 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dec5994-83a7-42be-bfaf-78c51d4bbd82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:38.438148 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-8dec5994-83a7-42be-bfaf-78c51d4bbd82 None None] MFA Rules not processed for user `82ea018632fb4487b081f23fca5b15a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:45:38.439399 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-8dec5994-83a7-42be-bfaf-78c51d4bbd82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:38.508220 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 996/2021] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:45:38 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:45:38.512883 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d5f9a9-2bd0-4697-8a6f-d6a90c0715a3 service neutron] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:38.524230 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d5f9a9-2bd0-4697-8a6f-d6a90c0715a3 service neutron] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:38.529245 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f6e18e-1ae3-4ea3-a295-b2ab4c34d7fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:38.530036 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f6e18e-1ae3-4ea3-a295-b2ab4c34d7fa admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:38.530723 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f6e18e-1ae3-4ea3-a295-b2ab4c34d7fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:38.531177 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f6e18e-1ae3-4ea3-a295-b2ab4c34d7fa admin admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:38.532280 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f6e18e-1ae3-4ea3-a295-b2ab4c34d7fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:38.534408 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f6e18e-1ae3-4ea3-a295-b2ab4c34d7fa admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:38.576247 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1026/2022] 10.222.0.89 () {60 vars in 1153 bytes} [Thu Jul 28 17:45:38 2022] POST /identity/v3/projects => generated 373 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:45:38.580577 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dec5994-83a7-42be-bfaf-78c51d4bbd82 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:38.592065 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dec5994-83a7-42be-bfaf-78c51d4bbd82 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:38.641794 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:45:38.659625 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9beae9-419c-4a2b-b975-ba39dc130629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:38.660275 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9beae9-419c-4a2b-b975-ba39dc130629 admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:38.660591 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9beae9-419c-4a2b-b975-ba39dc130629 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:38.660866 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9beae9-419c-4a2b-b975-ba39dc130629 admin admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:38.661619 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9beae9-419c-4a2b-b975-ba39dc130629 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:38.663533 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9beae9-419c-4a2b-b975-ba39dc130629 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:38.684803 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 997/2023] 10.222.0.89 () {60 vars in 1144 bytes} [Thu Jul 28 17:45:38 2022] POST /identity/v3/users => generated 293 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:45:38.689172 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f6e18e-1ae3-4ea3-a295-b2ab4c34d7fa admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:38.699633 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f6e18e-1ae3-4ea3-a295-b2ab4c34d7fa admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:38.705210 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa3dc77-b88c-4c70-9f41-5a2d2c7ede03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:38.705815 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa3dc77-b88c-4c70-9f41-5a2d2c7ede03 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:38.706099 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa3dc77-b88c-4c70-9f41-5a2d2c7ede03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:38.706372 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa3dc77-b88c-4c70-9f41-5a2d2c7ede03 admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:38.707066 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa3dc77-b88c-4c70-9f41-5a2d2c7ede03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:38.708652 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa3dc77-b88c-4c70-9f41-5a2d2c7ede03 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:38.720521 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1027/2024] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:45:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:45:38.724249 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9beae9-419c-4a2b-b975-ba39dc130629 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:38.734934 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9beae9-419c-4a2b-b975-ba39dc130629 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:38.739560 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067b6c0a-356c-4f8b-a970-ca1d06090c02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:38.740224 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067b6c0a-356c-4f8b-a970-ca1d06090c02 admin admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:38.740499 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067b6c0a-356c-4f8b-a970-ca1d06090c02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:38.740771 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067b6c0a-356c-4f8b-a970-ca1d06090c02 admin admin] PATH_INFO: `/v3/projects/8c0fae9988f9478f8eb6304a9fccbf27/users/486e02f5561c47c4b8bd99ecd6419d8f/roles/73e457231a824ea98c28d22610be640c` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:38.755449 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067b6c0a-356c-4f8b-a970-ca1d06090c02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c0fae9988f9478f8eb6304a9fccbf27, user_id=486e02f5561c47c4b8bd99ecd6419d8f, role_id=73e457231a824ea98c28d22610be640c)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:38.758180 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067b6c0a-356c-4f8b-a970-ca1d06090c02 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:38.773694 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 998/2025] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:45:38 2022] PUT /identity/v3/projects/8c0fae9988f9478f8eb6304a9fccbf27/users/486e02f5561c47c4b8bd99ecd6419d8f/roles/73e457231a824ea98c28d22610be640c => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 17:45:38.777622 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa3dc77-b88c-4c70-9f41-5a2d2c7ede03 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:38.788253 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa3dc77-b88c-4c70-9f41-5a2d2c7ede03 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:38.847235 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e871143c-5f44-4dcd-ad9b-25dbaf79992c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:38.847847 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e871143c-5f44-4dcd-ad9b-25dbaf79992c admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:38.848127 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e871143c-5f44-4dcd-ad9b-25dbaf79992c admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:38.848403 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e871143c-5f44-4dcd-ad9b-25dbaf79992c admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:38.849089 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e871143c-5f44-4dcd-ad9b-25dbaf79992c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:38.850710 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e871143c-5f44-4dcd-ad9b-25dbaf79992c admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:38.860451 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1028/2026] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:45:38 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:45:38.864803 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067b6c0a-356c-4f8b-a970-ca1d06090c02 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:38.875339 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067b6c0a-356c-4f8b-a970-ca1d06090c02 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:38.880451 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b49a08a-3873-4e8b-af82-99e60bcc32bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:38.881244 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b49a08a-3873-4e8b-af82-99e60bcc32bb admin admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:38.881559 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b49a08a-3873-4e8b-af82-99e60bcc32bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:38.881905 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b49a08a-3873-4e8b-af82-99e60bcc32bb admin admin] PATH_INFO: `/v3/projects/8c0fae9988f9478f8eb6304a9fccbf27/users/486e02f5561c47c4b8bd99ecd6419d8f/roles/73e457231a824ea98c28d22610be640c` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:38.883986 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b49a08a-3873-4e8b-af82-99e60bcc32bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c0fae9988f9478f8eb6304a9fccbf27, user_id=486e02f5561c47c4b8bd99ecd6419d8f, role_id=73e457231a824ea98c28d22610be640c)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:38.887052 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b49a08a-3873-4e8b-af82-99e60bcc32bb admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:38.898301 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 999/2027] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:45:38 2022] PUT /identity/v3/projects/8c0fae9988f9478f8eb6304a9fccbf27/users/486e02f5561c47c4b8bd99ecd6419d8f/roles/73e457231a824ea98c28d22610be640c => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:45:40.071629 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ad2659-a866-4a04-9129-daf1701257e4 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:40.072191 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ad2659-a866-4a04-9129-daf1701257e4 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:40.072301 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ad2659-a866-4a04-9129-daf1701257e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:40.106730 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-91ad2659-a866-4a04-9129-daf1701257e4 None None] MFA Rules not processed for user `486e02f5561c47c4b8bd99ecd6419d8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:45:40.108836 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-91ad2659-a866-4a04-9129-daf1701257e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:40.115263 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:45:40.203969 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1029/2028] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:45:40 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:45:40.215529 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b49a08a-3873-4e8b-af82-99e60bcc32bb admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:40.217344 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-6b49a08a-3873-4e8b-af82-99e60bcc32bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:40.279244 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b49a08a-3873-4e8b-af82-99e60bcc32bb admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:40.283185 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f28cc26-857d-46bd-96c0-de8c1a860f67 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96d512d0c84240918dae6c4a3e8b7b79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:40.283798 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f28cc26-857d-46bd-96c0-de8c1a860f67 service cinder] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:40.284081 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f28cc26-857d-46bd-96c0-de8c1a860f67 service cinder] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:40.284353 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f28cc26-857d-46bd-96c0-de8c1a860f67 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:40.340429 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f28cc26-857d-46bd-96c0-de8c1a860f67 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:40.341839 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f28cc26-857d-46bd-96c0-de8c1a860f67 service cinder] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:40.350867 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1000/2029] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:45:40 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:45:50.200991 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=26 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:45:50.325457 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ad2659-a866-4a04-9129-daf1701257e4 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:50.336007 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ad2659-a866-4a04-9129-daf1701257e4 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:50.339724 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d936d04b-ddd3-464a-b015-04d06b16784d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96d512d0c84240918dae6c4a3e8b7b79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:50.340321 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d936d04b-ddd3-464a-b015-04d06b16784d service cinder] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:50.340610 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d936d04b-ddd3-464a-b015-04d06b16784d service cinder] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:50.340868 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d936d04b-ddd3-464a-b015-04d06b16784d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:50.346739 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=72,keystone:INSERT=5 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:45:50.350816 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d936d04b-ddd3-464a-b015-04d06b16784d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:50.352288 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d936d04b-ddd3-464a-b015-04d06b16784d service cinder] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:50.362299 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1030/2030] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:45:50 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:45:52.392622 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f28cc26-857d-46bd-96c0-de8c1a860f67 service cinder] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:52.394532 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-3f28cc26-857d-46bd-96c0-de8c1a860f67 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:52.468768 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f28cc26-857d-46bd-96c0-de8c1a860f67 service cinder] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:52.474216 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b51d32a-52d9-4580-b778-07869809bf1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:52.475262 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b51d32a-52d9-4580-b778-07869809bf1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:52.475692 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b51d32a-52d9-4580-b778-07869809bf1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:52.476114 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b51d32a-52d9-4580-b778-07869809bf1b admin admin] PATH_INFO: `/v3/users/d6c606ccda304c4084a7150d5bd34e81` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:52.477493 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b51d32a-52d9-4580-b778-07869809bf1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=d6c606ccda304c4084a7150d5bd34e81)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:52.479769 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b51d32a-52d9-4580-b778-07869809bf1b admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:52.549038 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-2b51d32a-52d9-4580-b778-07869809bf1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd6c606ccda304c4084a7150d5bd34e81'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:52.558314 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-2b51d32a-52d9-4580-b778-07869809bf1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd6c606ccda304c4084a7150d5bd34e81'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:52.572208 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-2b51d32a-52d9-4580-b778-07869809bf1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd6c606ccda304c4084a7150d5bd34e81'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:52.573547 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-2b51d32a-52d9-4580-b778-07869809bf1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd6c606ccda304c4084a7150d5bd34e81'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:52.573729 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-2b51d32a-52d9-4580-b778-07869809bf1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd6c606ccda304c4084a7150d5bd34e81'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:52.593827 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1001/2031] 10.222.0.89 () {58 vars in 1224 bytes} [Thu Jul 28 17:45:52 2022] DELETE /identity/v3/users/d6c606ccda304c4084a7150d5bd34e81 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:45:52.598338 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7e25f3-b1dd-4c51-b009-fd8baa7bc0e8 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:52.598797 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7e25f3-b1dd-4c51-b009-fd8baa7bc0e8 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:52.598902 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7e25f3-b1dd-4c51-b009-fd8baa7bc0e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:52.619307 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-be7e25f3-b1dd-4c51-b009-fd8baa7bc0e8 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:45:52.620552 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-be7e25f3-b1dd-4c51-b009-fd8baa7bc0e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:52.692111 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1031/2032] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:45:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:45:52.701676 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b51d32a-52d9-4580-b778-07869809bf1b admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:52.703894 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-2b51d32a-52d9-4580-b778-07869809bf1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:52.767130 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b51d32a-52d9-4580-b778-07869809bf1b admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:52.770856 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3e974c-1ecd-4355-8aff-4c488b0c5ada service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76817cd541294331b0023069092e8e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:52.771477 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3e974c-1ecd-4355-8aff-4c488b0c5ada service neutron] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:52.771756 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3e974c-1ecd-4355-8aff-4c488b0c5ada service neutron] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:52.772032 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3e974c-1ecd-4355-8aff-4c488b0c5ada service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:52.781239 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3e974c-1ecd-4355-8aff-4c488b0c5ada service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:52.782660 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3e974c-1ecd-4355-8aff-4c488b0c5ada service neutron] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:52.792075 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1002/2033] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:45:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:45:53.096063 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7e25f3-b1dd-4c51-b009-fd8baa7bc0e8 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:53.097933 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-be7e25f3-b1dd-4c51-b009-fd8baa7bc0e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:53.114685 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7e25f3-b1dd-4c51-b009-fd8baa7bc0e8 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:53.123238 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec13d1a8-81a4-4295-903f-c1b343467c04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:53.126026 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec13d1a8-81a4-4295-903f-c1b343467c04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:53.126305 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec13d1a8-81a4-4295-903f-c1b343467c04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:53.127117 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec13d1a8-81a4-4295-903f-c1b343467c04 admin admin] PATH_INFO: `/v3/projects/dccaf4ea0ccd43fdb2d8a7a83cf1e94b` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:53.131258 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec13d1a8-81a4-4295-903f-c1b343467c04 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dccaf4ea0ccd43fdb2d8a7a83cf1e94b)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:53.133805 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec13d1a8-81a4-4295-903f-c1b343467c04 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:53.193410 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-ec13d1a8-81a4-4295-903f-c1b343467c04 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dccaf4ea0ccd43fdb2d8a7a83cf1e94b', 'request_id': 'req-ec13d1a8-81a4-4295-903f-c1b343467c04'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:53.198988 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-ec13d1a8-81a4-4295-903f-c1b343467c04 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dccaf4ea0ccd43fdb2d8a7a83cf1e94b', 'request_id': 'req-ec13d1a8-81a4-4295-903f-c1b343467c04'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:53.205864 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-ec13d1a8-81a4-4295-903f-c1b343467c04 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dccaf4ea0ccd43fdb2d8a7a83cf1e94b', 'request_id': 'req-ec13d1a8-81a4-4295-903f-c1b343467c04'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:53.219607 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-ec13d1a8-81a4-4295-903f-c1b343467c04 admin admin] The token cache is being invalidate because project dccaf4ea0ccd43fdb2d8a7a83cf1e94b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80324) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 17:45:53.219898 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-ec13d1a8-81a4-4295-903f-c1b343467c04 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:53.220525 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-ec13d1a8-81a4-4295-903f-c1b343467c04 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:53.222111 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1032/2034] 10.222.0.89 () {58 vars in 1233 bytes} [Thu Jul 28 17:45:53 2022] DELETE /identity/v3/projects/dccaf4ea0ccd43fdb2d8a7a83cf1e94b => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:45:55.218349 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3e974c-1ecd-4355-8aff-4c488b0c5ada service neutron] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:55.220490 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-4d3e974c-1ecd-4355-8aff-4c488b0c5ada service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:55.234108 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3e974c-1ecd-4355-8aff-4c488b0c5ada service neutron] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:55.258738 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:45:55.307970 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20695bd8-6d75-4c68-b2bb-19007b96bdc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:55.308632 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20695bd8-6d75-4c68-b2bb-19007b96bdc4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:55.309022 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20695bd8-6d75-4c68-b2bb-19007b96bdc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:55.309332 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20695bd8-6d75-4c68-b2bb-19007b96bdc4 admin admin] PATH_INFO: `/v3/users/6bfb0add8a7248e692f177ae58d782e3` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:55.310661 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20695bd8-6d75-4c68-b2bb-19007b96bdc4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6bfb0add8a7248e692f177ae58d782e3)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:55.313622 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20695bd8-6d75-4c68-b2bb-19007b96bdc4 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:55.366486 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-20695bd8-6d75-4c68-b2bb-19007b96bdc4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6bfb0add8a7248e692f177ae58d782e3'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:55.373365 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-20695bd8-6d75-4c68-b2bb-19007b96bdc4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6bfb0add8a7248e692f177ae58d782e3'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:55.382695 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-20695bd8-6d75-4c68-b2bb-19007b96bdc4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6bfb0add8a7248e692f177ae58d782e3'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:55.383532 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-20695bd8-6d75-4c68-b2bb-19007b96bdc4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6bfb0add8a7248e692f177ae58d782e3'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:55.384449 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-20695bd8-6d75-4c68-b2bb-19007b96bdc4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6bfb0add8a7248e692f177ae58d782e3'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:55.406902 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1003/2035] 10.222.0.89 () {58 vars in 1224 bytes} [Thu Jul 28 17:45:55 2022] DELETE /identity/v3/users/6bfb0add8a7248e692f177ae58d782e3 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:45:55.411988 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060d184e-2810-49ff-90ce-83f345a8f6dd None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:55.412522 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060d184e-2810-49ff-90ce-83f345a8f6dd None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:55.412522 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060d184e-2810-49ff-90ce-83f345a8f6dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:55.434515 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-060d184e-2810-49ff-90ce-83f345a8f6dd None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:45:55.435872 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-060d184e-2810-49ff-90ce-83f345a8f6dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:55.518151 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1033/2036] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:45:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:45:55.531964 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20695bd8-6d75-4c68-b2bb-19007b96bdc4 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:55.533650 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-20695bd8-6d75-4c68-b2bb-19007b96bdc4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:55.605548 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20695bd8-6d75-4c68-b2bb-19007b96bdc4 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:55.613629 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181189c7-e86d-408c-ac47-e061b44614b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76817cd541294331b0023069092e8e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:55.614756 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181189c7-e86d-408c-ac47-e061b44614b9 service neutron] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:55.615111 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181189c7-e86d-408c-ac47-e061b44614b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:55.615679 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181189c7-e86d-408c-ac47-e061b44614b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:55.629520 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181189c7-e86d-408c-ac47-e061b44614b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:55.631149 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181189c7-e86d-408c-ac47-e061b44614b9 service neutron] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:55.649550 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1004/2037] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:45:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 17:45:56.185025 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060d184e-2810-49ff-90ce-83f345a8f6dd None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:56.187288 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-060d184e-2810-49ff-90ce-83f345a8f6dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:56.209440 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060d184e-2810-49ff-90ce-83f345a8f6dd None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:56.220516 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db098269-c647-4d48-8555-9258fcfd871f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:56.221303 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db098269-c647-4d48-8555-9258fcfd871f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:56.221648 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db098269-c647-4d48-8555-9258fcfd871f admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:56.222125 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db098269-c647-4d48-8555-9258fcfd871f admin admin] PATH_INFO: `/v3/projects/67c07aa592e24366b3c9d844ea20085b` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:56.223646 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db098269-c647-4d48-8555-9258fcfd871f admin admin] RBAC: Authorizing `identity:delete_project(project_id=67c07aa592e24366b3c9d844ea20085b)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:56.226123 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db098269-c647-4d48-8555-9258fcfd871f admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:56.242428 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:45:56.318944 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-db098269-c647-4d48-8555-9258fcfd871f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67c07aa592e24366b3c9d844ea20085b', 'request_id': 'req-db098269-c647-4d48-8555-9258fcfd871f'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:56.324420 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-db098269-c647-4d48-8555-9258fcfd871f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67c07aa592e24366b3c9d844ea20085b', 'request_id': 'req-db098269-c647-4d48-8555-9258fcfd871f'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:56.331242 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-db098269-c647-4d48-8555-9258fcfd871f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67c07aa592e24366b3c9d844ea20085b', 'request_id': 'req-db098269-c647-4d48-8555-9258fcfd871f'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:56.344343 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-db098269-c647-4d48-8555-9258fcfd871f admin admin] The token cache is being invalidate because project 67c07aa592e24366b3c9d844ea20085b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80324) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 17:45:56.344646 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-db098269-c647-4d48-8555-9258fcfd871f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:56.345305 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-db098269-c647-4d48-8555-9258fcfd871f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:56.346832 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1034/2038] 10.222.0.89 () {58 vars in 1233 bytes} [Thu Jul 28 17:45:56 2022] DELETE /identity/v3/projects/67c07aa592e24366b3c9d844ea20085b => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:45:56.351021 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181189c7-e86d-408c-ac47-e061b44614b9 service neutron] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:56.363289 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181189c7-e86d-408c-ac47-e061b44614b9 service neutron] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:56.412698 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:45:56.432282 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e51609-5538-49ee-8643-48ae1096a05c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:56.432939 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e51609-5538-49ee-8643-48ae1096a05c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:56.433244 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e51609-5538-49ee-8643-48ae1096a05c admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:56.433541 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e51609-5538-49ee-8643-48ae1096a05c admin admin] PATH_INFO: `/v3/users/9a144c2042054e4fa67614033924c06d` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:56.434719 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e51609-5538-49ee-8643-48ae1096a05c admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a144c2042054e4fa67614033924c06d)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:56.438596 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e51609-5538-49ee-8643-48ae1096a05c admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:56.498527 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-e2e51609-5538-49ee-8643-48ae1096a05c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a144c2042054e4fa67614033924c06d'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:56.506155 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-e2e51609-5538-49ee-8643-48ae1096a05c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a144c2042054e4fa67614033924c06d'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:56.514019 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-e2e51609-5538-49ee-8643-48ae1096a05c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a144c2042054e4fa67614033924c06d'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:56.514753 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-e2e51609-5538-49ee-8643-48ae1096a05c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a144c2042054e4fa67614033924c06d'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:56.515374 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-e2e51609-5538-49ee-8643-48ae1096a05c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a144c2042054e4fa67614033924c06d'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:56.532544 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1005/2039] 10.222.0.89 () {58 vars in 1224 bytes} [Thu Jul 28 17:45:56 2022] DELETE /identity/v3/users/9a144c2042054e4fa67614033924c06d => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:45:56.842367 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db098269-c647-4d48-8555-9258fcfd871f admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:56.845823 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-db098269-c647-4d48-8555-9258fcfd871f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:56.866603 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db098269-c647-4d48-8555-9258fcfd871f admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:56.947770 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887845a8-5b02-498e-8147-689be3a4daf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:56.948458 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887845a8-5b02-498e-8147-689be3a4daf8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:56.948744 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887845a8-5b02-498e-8147-689be3a4daf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:56.949038 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887845a8-5b02-498e-8147-689be3a4daf8 admin admin] PATH_INFO: `/v3/projects/fc80d51e20d5477ca93fb98d6815a048` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:56.950327 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887845a8-5b02-498e-8147-689be3a4daf8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc80d51e20d5477ca93fb98d6815a048)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:56.952461 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887845a8-5b02-498e-8147-689be3a4daf8 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:57.020726 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-887845a8-5b02-498e-8147-689be3a4daf8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc80d51e20d5477ca93fb98d6815a048', 'request_id': 'req-887845a8-5b02-498e-8147-689be3a4daf8'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:57.027648 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-887845a8-5b02-498e-8147-689be3a4daf8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc80d51e20d5477ca93fb98d6815a048', 'request_id': 'req-887845a8-5b02-498e-8147-689be3a4daf8'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:57.036721 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-887845a8-5b02-498e-8147-689be3a4daf8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc80d51e20d5477ca93fb98d6815a048', 'request_id': 'req-887845a8-5b02-498e-8147-689be3a4daf8'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:57.051908 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-887845a8-5b02-498e-8147-689be3a4daf8 admin admin] The token cache is being invalidate because project fc80d51e20d5477ca93fb98d6815a048 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80324) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 17:45:57.052173 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-887845a8-5b02-498e-8147-689be3a4daf8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:57.052967 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-887845a8-5b02-498e-8147-689be3a4daf8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:45:57.054766 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1035/2040] 10.222.0.89 () {58 vars in 1233 bytes} [Thu Jul 28 17:45:56 2022] DELETE /identity/v3/projects/fc80d51e20d5477ca93fb98d6815a048 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:45:57.059944 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee6f9fa-aaba-4913-9564-541c1e3386e6 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:57.060237 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee6f9fa-aaba-4913-9564-541c1e3386e6 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:57.060520 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee6f9fa-aaba-4913-9564-541c1e3386e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:57.083251 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-6ee6f9fa-aaba-4913-9564-541c1e3386e6 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:45:57.085325 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-6ee6f9fa-aaba-4913-9564-541c1e3386e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:57.159695 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1006/2041] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:45:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:45:57.169351 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcfcde6-9bbd-416c-83d6-af140c49b151 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:57.169351 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcfcde6-9bbd-416c-83d6-af140c49b151 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:57.169351 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcfcde6-9bbd-416c-83d6-af140c49b151 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:57.191828 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-afcfcde6-9bbd-416c-83d6-af140c49b151 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:45:57.193019 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-afcfcde6-9bbd-416c-83d6-af140c49b151 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:57.201372 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1036/2042] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:45:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:45:57.205775 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee6f9fa-aaba-4913-9564-541c1e3386e6 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:57.215817 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee6f9fa-aaba-4913-9564-541c1e3386e6 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:57.221703 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a459a69d-0347-4875-aec0-35d60b237ffb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:57.222324 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a459a69d-0347-4875-aec0-35d60b237ffb admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:57.222609 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a459a69d-0347-4875-aec0-35d60b237ffb admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:57.222888 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a459a69d-0347-4875-aec0-35d60b237ffb admin admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:57.223593 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a459a69d-0347-4875-aec0-35d60b237ffb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:45:57.223923 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a459a69d-0347-4875-aec0-35d60b237ffb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:57.225570 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a459a69d-0347-4875-aec0-35d60b237ffb admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:57.238047 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1007/2043] 10.222.0.89 () {58 vars in 1153 bytes} [Thu Jul 28 17:45:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:45:57.242462 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcfcde6-9bbd-416c-83d6-af140c49b151 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:57.253077 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcfcde6-9bbd-416c-83d6-af140c49b151 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:57.258132 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4386d3-7466-4786-9e01-79934d832455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:57.258736 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4386d3-7466-4786-9e01-79934d832455 admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:57.259012 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4386d3-7466-4786-9e01-79934d832455 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:57.259280 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4386d3-7466-4786-9e01-79934d832455 admin admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:57.259960 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4386d3-7466-4786-9e01-79934d832455 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:57.262191 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4386d3-7466-4786-9e01-79934d832455 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:57.277958 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:45:57.295302 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1037/2044] 10.222.0.89 () {60 vars in 1153 bytes} [Thu Jul 28 17:45:57 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:45:57.297526 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a459a69d-0347-4875-aec0-35d60b237ffb admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:57.307741 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a459a69d-0347-4875-aec0-35d60b237ffb admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:57.360254 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:45:57.373120 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afb53cd-8a7f-46af-b326-884a2d88b456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:57.374423 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afb53cd-8a7f-46af-b326-884a2d88b456 admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:57.374694 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afb53cd-8a7f-46af-b326-884a2d88b456 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:57.374893 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afb53cd-8a7f-46af-b326-884a2d88b456 admin admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:57.375766 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afb53cd-8a7f-46af-b326-884a2d88b456 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:57.377887 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afb53cd-8a7f-46af-b326-884a2d88b456 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:57.406652 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1008/2045] 10.222.0.89 () {60 vars in 1144 bytes} [Thu Jul 28 17:45:57 2022] POST /identity/v3/users => generated 287 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 17:45:57.407947 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4386d3-7466-4786-9e01-79934d832455 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:57.419275 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4386d3-7466-4786-9e01-79934d832455 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:57.424390 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081f7dad-5ef6-402f-8bd9-ee2e2e5c85ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:57.424991 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081f7dad-5ef6-402f-8bd9-ee2e2e5c85ec admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:57.425326 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081f7dad-5ef6-402f-8bd9-ee2e2e5c85ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:57.425579 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081f7dad-5ef6-402f-8bd9-ee2e2e5c85ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:57.426276 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081f7dad-5ef6-402f-8bd9-ee2e2e5c85ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:57.427928 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081f7dad-5ef6-402f-8bd9-ee2e2e5c85ec admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:57.438457 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1038/2046] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:45:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:45:57.442553 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afb53cd-8a7f-46af-b326-884a2d88b456 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:57.454640 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afb53cd-8a7f-46af-b326-884a2d88b456 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:57.460246 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7887594e-2106-443f-9517-f9ce2f61b826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:57.461137 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7887594e-2106-443f-9517-f9ce2f61b826 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:57.461562 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7887594e-2106-443f-9517-f9ce2f61b826 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:57.461944 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7887594e-2106-443f-9517-f9ce2f61b826 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:57.463110 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7887594e-2106-443f-9517-f9ce2f61b826 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:57.465115 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7887594e-2106-443f-9517-f9ce2f61b826 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:57.477189 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1009/2047] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:45:57 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:45:57.481431 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081f7dad-5ef6-402f-8bd9-ee2e2e5c85ec admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:57.491968 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081f7dad-5ef6-402f-8bd9-ee2e2e5c85ec admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:57.497301 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003652df-ab58-4cfe-baf9-74ff053ed71b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:57.497964 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003652df-ab58-4cfe-baf9-74ff053ed71b admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:57.498243 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003652df-ab58-4cfe-baf9-74ff053ed71b admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:57.498517 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003652df-ab58-4cfe-baf9-74ff053ed71b admin admin] PATH_INFO: `/v3/projects/72a2828ca9f14497842a061badda17fd/users/52286cfb8c4641a394371115bef41532/roles/11f8731756fe4caca13efa61b71b11c5` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:57.515405 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003652df-ab58-4cfe-baf9-74ff053ed71b admin admin] RBAC: Authorizing `identity:create_grant(project_id=72a2828ca9f14497842a061badda17fd, user_id=52286cfb8c4641a394371115bef41532, role_id=11f8731756fe4caca13efa61b71b11c5)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:57.518152 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003652df-ab58-4cfe-baf9-74ff053ed71b admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:57.531621 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1039/2048] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:45:57 2022] PUT /identity/v3/projects/72a2828ca9f14497842a061badda17fd/users/52286cfb8c4641a394371115bef41532/roles/11f8731756fe4caca13efa61b71b11c5 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:45:57.541879 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0736687-e60d-49be-8a9a-649785fec5c4 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:57.542203 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0736687-e60d-49be-8a9a-649785fec5c4 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:57.542582 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0736687-e60d-49be-8a9a-649785fec5c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:57.571884 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-c0736687-e60d-49be-8a9a-649785fec5c4 None None] MFA Rules not processed for user `52286cfb8c4641a394371115bef41532`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:45:57.573272 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-c0736687-e60d-49be-8a9a-649785fec5c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:57.642368 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1010/2049] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:45:57 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:45:57.648531 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003652df-ab58-4cfe-baf9-74ff053ed71b admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:57.718583 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003652df-ab58-4cfe-baf9-74ff053ed71b admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:57.723524 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8186f082-2214-4ff6-99ac-8fbefef053d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:57.724137 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8186f082-2214-4ff6-99ac-8fbefef053d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:57.724421 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8186f082-2214-4ff6-99ac-8fbefef053d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:57.724694 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8186f082-2214-4ff6-99ac-8fbefef053d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:57.725644 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8186f082-2214-4ff6-99ac-8fbefef053d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:57.727998 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8186f082-2214-4ff6-99ac-8fbefef053d0 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:57.756896 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1040/2050] 10.222.0.89 () {60 vars in 1153 bytes} [Thu Jul 28 17:45:57 2022] POST /identity/v3/projects => generated 375 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:45:57.761873 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0736687-e60d-49be-8a9a-649785fec5c4 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:57.773723 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0736687-e60d-49be-8a9a-649785fec5c4 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:57.832904 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e957e0f-4095-4518-addb-04e69afd5c3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:57.833571 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e957e0f-4095-4518-addb-04e69afd5c3d admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:57.833859 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e957e0f-4095-4518-addb-04e69afd5c3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:57.834134 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e957e0f-4095-4518-addb-04e69afd5c3d admin admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:57.834803 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e957e0f-4095-4518-addb-04e69afd5c3d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:57.836722 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e957e0f-4095-4518-addb-04e69afd5c3d admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:57.860988 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1011/2051] 10.222.0.89 () {60 vars in 1144 bytes} [Thu Jul 28 17:45:57 2022] POST /identity/v3/users => generated 294 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:45:57.865024 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8186f082-2214-4ff6-99ac-8fbefef053d0 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:57.875588 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8186f082-2214-4ff6-99ac-8fbefef053d0 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:57.880207 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd10e07b-3434-4cb0-bd83-0be261f6acc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:57.880810 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd10e07b-3434-4cb0-bd83-0be261f6acc1 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:57.881090 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd10e07b-3434-4cb0-bd83-0be261f6acc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:57.881467 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd10e07b-3434-4cb0-bd83-0be261f6acc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:57.882191 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd10e07b-3434-4cb0-bd83-0be261f6acc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:57.883776 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd10e07b-3434-4cb0-bd83-0be261f6acc1 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:57.895646 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1041/2052] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:45:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:45:57.900040 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e957e0f-4095-4518-addb-04e69afd5c3d admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:57.911080 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e957e0f-4095-4518-addb-04e69afd5c3d admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:57.915775 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3a4583-edad-4630-8767-e4b30631fbd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:57.916444 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3a4583-edad-4630-8767-e4b30631fbd3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:57.916723 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3a4583-edad-4630-8767-e4b30631fbd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:57.916996 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3a4583-edad-4630-8767-e4b30631fbd3 admin admin] PATH_INFO: `/v3/projects/8a7b4302ded54ebb85a507170e28d68a/users/6536fa39890a48babe9df5598ddb66a2/roles/73e457231a824ea98c28d22610be640c` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:57.931900 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3a4583-edad-4630-8767-e4b30631fbd3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a7b4302ded54ebb85a507170e28d68a, user_id=6536fa39890a48babe9df5598ddb66a2, role_id=73e457231a824ea98c28d22610be640c)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:57.934587 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3a4583-edad-4630-8767-e4b30631fbd3 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:57.947786 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1012/2053] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:45:57 2022] PUT /identity/v3/projects/8a7b4302ded54ebb85a507170e28d68a/users/6536fa39890a48babe9df5598ddb66a2/roles/73e457231a824ea98c28d22610be640c => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:45:57.953091 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd10e07b-3434-4cb0-bd83-0be261f6acc1 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:57.963768 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd10e07b-3434-4cb0-bd83-0be261f6acc1 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:58.020784 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f54a48e-0123-4144-8586-9e8b38d3424a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:58.021429 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f54a48e-0123-4144-8586-9e8b38d3424a admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:58.021715 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f54a48e-0123-4144-8586-9e8b38d3424a admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:58.021991 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f54a48e-0123-4144-8586-9e8b38d3424a admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:58.022680 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f54a48e-0123-4144-8586-9e8b38d3424a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:58.024347 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f54a48e-0123-4144-8586-9e8b38d3424a admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:58.034575 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1042/2054] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:45:57 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:45:58.038885 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3a4583-edad-4630-8767-e4b30631fbd3 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:58.049636 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3a4583-edad-4630-8767-e4b30631fbd3 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:58.054266 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9080637d-d548-41da-8244-5ee93c9d1f8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:58.054922 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9080637d-d548-41da-8244-5ee93c9d1f8f admin admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:58.055205 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9080637d-d548-41da-8244-5ee93c9d1f8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:58.055480 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9080637d-d548-41da-8244-5ee93c9d1f8f admin admin] PATH_INFO: `/v3/projects/8a7b4302ded54ebb85a507170e28d68a/users/6536fa39890a48babe9df5598ddb66a2/roles/73e457231a824ea98c28d22610be640c` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:58.057454 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9080637d-d548-41da-8244-5ee93c9d1f8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a7b4302ded54ebb85a507170e28d68a, user_id=6536fa39890a48babe9df5598ddb66a2, role_id=73e457231a824ea98c28d22610be640c)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:58.060085 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9080637d-d548-41da-8244-5ee93c9d1f8f admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:58.071480 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1013/2055] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:45:58 2022] PUT /identity/v3/projects/8a7b4302ded54ebb85a507170e28d68a/users/6536fa39890a48babe9df5598ddb66a2/roles/73e457231a824ea98c28d22610be640c => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 17:45:58.083594 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3963a9ee-75ef-46b2-a193-d5a751a5486c None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:58.083928 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3963a9ee-75ef-46b2-a193-d5a751a5486c None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:58.084152 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3963a9ee-75ef-46b2-a193-d5a751a5486c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:58.113255 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-3963a9ee-75ef-46b2-a193-d5a751a5486c None None] MFA Rules not processed for user `6536fa39890a48babe9df5598ddb66a2`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:45:58.114493 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-3963a9ee-75ef-46b2-a193-d5a751a5486c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:58.133631 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:45:58.188740 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1043/2056] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:45:58 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:45:58.204804 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9080637d-d548-41da-8244-5ee93c9d1f8f admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:45:58.206962 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-9080637d-d548-41da-8244-5ee93c9d1f8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:45:58.224389 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:45:58.286152 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9080637d-d548-41da-8244-5ee93c9d1f8f admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:45:58.293199 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788b9b2e-39e7-4613-aa78-1ade359c269b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96d512d0c84240918dae6c4a3e8b7b79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:45:58.293845 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788b9b2e-39e7-4613-aa78-1ade359c269b service cinder] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:45:58.294128 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788b9b2e-39e7-4613-aa78-1ade359c269b service cinder] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:45:58.294405 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788b9b2e-39e7-4613-aa78-1ade359c269b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:45:58.367867 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788b9b2e-39e7-4613-aa78-1ade359c269b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:45:58.369501 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788b9b2e-39e7-4613-aa78-1ade359c269b service cinder] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:45:58.380116 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1014/2057] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:45:58 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:46:04.183091 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3963a9ee-75ef-46b2-a193-d5a751a5486c None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:04.184811 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-3963a9ee-75ef-46b2-a193-d5a751a5486c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:46:04.197596 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3963a9ee-75ef-46b2-a193-d5a751a5486c None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:04.203266 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a27998c-fc8b-4c92-a71e-dbf24b848cde service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96d512d0c84240918dae6c4a3e8b7b79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:04.203266 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a27998c-fc8b-4c92-a71e-dbf24b848cde service cinder] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:04.203266 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a27998c-fc8b-4c92-a71e-dbf24b848cde service cinder] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:04.203266 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a27998c-fc8b-4c92-a71e-dbf24b848cde service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:04.212080 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a27998c-fc8b-4c92-a71e-dbf24b848cde service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:04.213524 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a27998c-fc8b-4c92-a71e-dbf24b848cde service cinder] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:04.223948 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1044/2058] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:46:04 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:46:08.374967 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=43 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:46:10.520367 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1950e0ca-4020-4712-8988-6dd6aea7f20b None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:10.520975 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1950e0ca-4020-4712-8988-6dd6aea7f20b None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:10.520975 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1950e0ca-4020-4712-8988-6dd6aea7f20b None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:10.522776 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1015/2059] 10.222.0.89 () {54 vars in 921 bytes} [Thu Jul 28 17:46:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:46:10.549594 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347c223f-9306-4bb7-a7c1-dd6ce12680c2 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:10.550159 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347c223f-9306-4bb7-a7c1-dd6ce12680c2 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:10.550159 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347c223f-9306-4bb7-a7c1-dd6ce12680c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:10.573945 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-347c223f-9306-4bb7-a7c1-dd6ce12680c2 None None] MFA Rules not processed for user `e4bd290018394b0d955cd376ce35b77f`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:46:10.575468 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-347c223f-9306-4bb7-a7c1-dd6ce12680c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:46:10.588204 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1950e0ca-4020-4712-8988-6dd6aea7f20b None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:10.590092 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-1950e0ca-4020-4712-8988-6dd6aea7f20b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:46:10.667611 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1045/2060] 10.222.0.89 () {58 vars in 1018 bytes} [Thu Jul 28 17:46:10 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:46:10.680353 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347c223f-9306-4bb7-a7c1-dd6ce12680c2 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:10.681830 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-347c223f-9306-4bb7-a7c1-dd6ce12680c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:46:10.696616 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347c223f-9306-4bb7-a7c1-dd6ce12680c2 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:10.697699 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1950e0ca-4020-4712-8988-6dd6aea7f20b None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:10.703276 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bc9738-538f-4e31-872d-70c51eadd73d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4bd290018394b0d955cd376ce35b77f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:10.703959 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bc9738-538f-4e31-872d-70c51eadd73d service placement] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:10.704242 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bc9738-538f-4e31-872d-70c51eadd73d service placement] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:10.704511 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bc9738-538f-4e31-872d-70c51eadd73d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:10.708582 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04928753-cf36-43bd-b1f1-2642a477183a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:10.709270 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04928753-cf36-43bd-b1f1-2642a477183a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:10.709556 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04928753-cf36-43bd-b1f1-2642a477183a admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:10.709832 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04928753-cf36-43bd-b1f1-2642a477183a admin admin] PATH_INFO: `/v3/users/82ea018632fb4487b081f23fca5b15a3` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:10.711764 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04928753-cf36-43bd-b1f1-2642a477183a admin admin] RBAC: Authorizing `identity:delete_user(user_id=82ea018632fb4487b081f23fca5b15a3)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:10.713596 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bc9738-538f-4e31-872d-70c51eadd73d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:10.713776 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04928753-cf36-43bd-b1f1-2642a477183a admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:10.715310 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bc9738-538f-4e31-872d-70c51eadd73d service placement] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:10.727421 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1046/2061] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:46:10 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:46:10.771723 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-04928753-cf36-43bd-b1f1-2642a477183a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82ea018632fb4487b081f23fca5b15a3'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:10.783095 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-04928753-cf36-43bd-b1f1-2642a477183a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82ea018632fb4487b081f23fca5b15a3'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:10.791177 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-04928753-cf36-43bd-b1f1-2642a477183a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82ea018632fb4487b081f23fca5b15a3'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:10.791843 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-04928753-cf36-43bd-b1f1-2642a477183a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82ea018632fb4487b081f23fca5b15a3'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:10.792480 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-04928753-cf36-43bd-b1f1-2642a477183a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82ea018632fb4487b081f23fca5b15a3'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:10.818471 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1016/2062] 10.222.0.89 () {58 vars in 1224 bytes} [Thu Jul 28 17:46:10 2022] DELETE /identity/v3/users/82ea018632fb4487b081f23fca5b15a3 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:46:11.252427 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bc9738-538f-4e31-872d-70c51eadd73d service placement] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:11.269711 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bc9738-538f-4e31-872d-70c51eadd73d service placement] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:11.388613 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a19cca-9061-406b-bcd1-e6f6db7354f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:11.392188 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a19cca-9061-406b-bcd1-e6f6db7354f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:11.392188 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a19cca-9061-406b-bcd1-e6f6db7354f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:11.392188 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a19cca-9061-406b-bcd1-e6f6db7354f1 admin admin] PATH_INFO: `/v3/projects/6bd0a6d50b6345339f0e6adccd157202` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:11.394009 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a19cca-9061-406b-bcd1-e6f6db7354f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6bd0a6d50b6345339f0e6adccd157202)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:11.396000 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a19cca-9061-406b-bcd1-e6f6db7354f1 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:11.433049 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:46:11.459881 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-f3a19cca-9061-406b-bcd1-e6f6db7354f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6bd0a6d50b6345339f0e6adccd157202', 'request_id': 'req-f3a19cca-9061-406b-bcd1-e6f6db7354f1'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:11.469095 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-f3a19cca-9061-406b-bcd1-e6f6db7354f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6bd0a6d50b6345339f0e6adccd157202', 'request_id': 'req-f3a19cca-9061-406b-bcd1-e6f6db7354f1'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:11.480154 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-f3a19cca-9061-406b-bcd1-e6f6db7354f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6bd0a6d50b6345339f0e6adccd157202', 'request_id': 'req-f3a19cca-9061-406b-bcd1-e6f6db7354f1'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:11.495548 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-f3a19cca-9061-406b-bcd1-e6f6db7354f1 admin admin] The token cache is being invalidate because project 6bd0a6d50b6345339f0e6adccd157202 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80324) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 17:46:11.495961 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-f3a19cca-9061-406b-bcd1-e6f6db7354f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:11.496583 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-f3a19cca-9061-406b-bcd1-e6f6db7354f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:11.498255 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1047/2063] 10.222.0.89 () {58 vars in 1233 bytes} [Thu Jul 28 17:46:11 2022] DELETE /identity/v3/projects/6bd0a6d50b6345339f0e6adccd157202 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:46:11.503283 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04928753-cf36-43bd-b1f1-2642a477183a admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:11.504700 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-04928753-cf36-43bd-b1f1-2642a477183a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:46:11.518333 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04928753-cf36-43bd-b1f1-2642a477183a admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:11.581981 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a20b4d-9b1a-4411-bf62-307bf7053dfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:11.582762 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a20b4d-9b1a-4411-bf62-307bf7053dfe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:11.582965 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a20b4d-9b1a-4411-bf62-307bf7053dfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:11.583216 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a20b4d-9b1a-4411-bf62-307bf7053dfe admin admin] PATH_INFO: `/v3/users/486e02f5561c47c4b8bd99ecd6419d8f` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:11.584479 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a20b4d-9b1a-4411-bf62-307bf7053dfe admin admin] RBAC: Authorizing `identity:delete_user(user_id=486e02f5561c47c4b8bd99ecd6419d8f)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:11.586500 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a20b4d-9b1a-4411-bf62-307bf7053dfe admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:11.612507 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:46:11.644689 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-02a20b4d-9b1a-4411-bf62-307bf7053dfe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '486e02f5561c47c4b8bd99ecd6419d8f'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:11.653196 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-02a20b4d-9b1a-4411-bf62-307bf7053dfe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '486e02f5561c47c4b8bd99ecd6419d8f'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:11.662393 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-02a20b4d-9b1a-4411-bf62-307bf7053dfe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '486e02f5561c47c4b8bd99ecd6419d8f'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:11.663118 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-02a20b4d-9b1a-4411-bf62-307bf7053dfe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '486e02f5561c47c4b8bd99ecd6419d8f'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:11.663900 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-02a20b4d-9b1a-4411-bf62-307bf7053dfe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '486e02f5561c47c4b8bd99ecd6419d8f'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:11.683470 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1017/2064] 10.222.0.89 () {58 vars in 1224 bytes} [Thu Jul 28 17:46:11 2022] DELETE /identity/v3/users/486e02f5561c47c4b8bd99ecd6419d8f => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:46:11.878718 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a19cca-9061-406b-bcd1-e6f6db7354f1 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:11.880358 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-f3a19cca-9061-406b-bcd1-e6f6db7354f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:46:11.895488 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a19cca-9061-406b-bcd1-e6f6db7354f1 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:11.965739 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3c22b2-8bb5-4087-9af6-d36152f1b930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:11.966381 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3c22b2-8bb5-4087-9af6-d36152f1b930 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:11.966659 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3c22b2-8bb5-4087-9af6-d36152f1b930 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:11.966933 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3c22b2-8bb5-4087-9af6-d36152f1b930 admin admin] PATH_INFO: `/v3/projects/8c0fae9988f9478f8eb6304a9fccbf27` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:11.968658 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3c22b2-8bb5-4087-9af6-d36152f1b930 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c0fae9988f9478f8eb6304a9fccbf27)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:11.970607 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3c22b2-8bb5-4087-9af6-d36152f1b930 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:12.031297 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-de3c22b2-8bb5-4087-9af6-d36152f1b930 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c0fae9988f9478f8eb6304a9fccbf27', 'request_id': 'req-de3c22b2-8bb5-4087-9af6-d36152f1b930'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:12.041363 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-de3c22b2-8bb5-4087-9af6-d36152f1b930 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c0fae9988f9478f8eb6304a9fccbf27', 'request_id': 'req-de3c22b2-8bb5-4087-9af6-d36152f1b930'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:12.053684 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-de3c22b2-8bb5-4087-9af6-d36152f1b930 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c0fae9988f9478f8eb6304a9fccbf27', 'request_id': 'req-de3c22b2-8bb5-4087-9af6-d36152f1b930'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:12.078453 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-de3c22b2-8bb5-4087-9af6-d36152f1b930 admin admin] The token cache is being invalidate because project 8c0fae9988f9478f8eb6304a9fccbf27 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80324) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 17:46:12.078788 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-de3c22b2-8bb5-4087-9af6-d36152f1b930 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:12.079644 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-de3c22b2-8bb5-4087-9af6-d36152f1b930 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:12.082173 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1048/2065] 10.222.0.89 () {58 vars in 1233 bytes} [Thu Jul 28 17:46:11 2022] DELETE /identity/v3/projects/8c0fae9988f9478f8eb6304a9fccbf27 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:46:12.089144 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703e2f8c-d7d3-4c3a-ae63-08aa73fab34b None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:12.089508 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703e2f8c-d7d3-4c3a-ae63-08aa73fab34b None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:12.089780 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703e2f8c-d7d3-4c3a-ae63-08aa73fab34b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:12.114017 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-703e2f8c-d7d3-4c3a-ae63-08aa73fab34b None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:46:12.115349 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-703e2f8c-d7d3-4c3a-ae63-08aa73fab34b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:46:12.192150 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1018/2066] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:46:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 17:46:12.199825 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2e7565-02bd-4e85-ba0a-69b6c3228e9a None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:12.200121 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2e7565-02bd-4e85-ba0a-69b6c3228e9a None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:12.200411 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2e7565-02bd-4e85-ba0a-69b6c3228e9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:12.225069 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-df2e7565-02bd-4e85-ba0a-69b6c3228e9a None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:46:12.226321 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-df2e7565-02bd-4e85-ba0a-69b6c3228e9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:46:12.235129 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1049/2067] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:46:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:46:12.239385 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703e2f8c-d7d3-4c3a-ae63-08aa73fab34b None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:12.249639 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703e2f8c-d7d3-4c3a-ae63-08aa73fab34b None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:12.255191 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782976ce-d02c-452a-8c72-bd0c9da38980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:12.256299 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782976ce-d02c-452a-8c72-bd0c9da38980 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:12.256582 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782976ce-d02c-452a-8c72-bd0c9da38980 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:12.256854 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782976ce-d02c-452a-8c72-bd0c9da38980 admin admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:12.258029 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782976ce-d02c-452a-8c72-bd0c9da38980 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:46:12.258483 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782976ce-d02c-452a-8c72-bd0c9da38980 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:12.260137 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782976ce-d02c-452a-8c72-bd0c9da38980 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:12.273327 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1019/2068] 10.222.0.89 () {58 vars in 1153 bytes} [Thu Jul 28 17:46:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:46:12.277275 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2e7565-02bd-4e85-ba0a-69b6c3228e9a None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:12.287185 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2e7565-02bd-4e85-ba0a-69b6c3228e9a None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:12.292015 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a73107-de93-4778-853a-b932962b8ece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:12.292641 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a73107-de93-4778-853a-b932962b8ece admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:12.292917 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a73107-de93-4778-853a-b932962b8ece admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:12.293227 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a73107-de93-4778-853a-b932962b8ece admin admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:12.293899 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a73107-de93-4778-853a-b932962b8ece admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:12.296318 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a73107-de93-4778-853a-b932962b8ece admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:12.326087 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1050/2069] 10.222.0.89 () {60 vars in 1153 bytes} [Thu Jul 28 17:46:12 2022] POST /identity/v3/projects => generated 371 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:46:12.330220 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782976ce-d02c-452a-8c72-bd0c9da38980 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:12.340407 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782976ce-d02c-452a-8c72-bd0c9da38980 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:12.399510 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ce3f81-4acd-4eaf-a627-2723340546e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:12.400146 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ce3f81-4acd-4eaf-a627-2723340546e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:12.400425 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ce3f81-4acd-4eaf-a627-2723340546e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:12.400709 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ce3f81-4acd-4eaf-a627-2723340546e5 admin admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:12.402059 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ce3f81-4acd-4eaf-a627-2723340546e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:12.404000 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ce3f81-4acd-4eaf-a627-2723340546e5 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:12.430826 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1020/2070] 10.222.0.89 () {60 vars in 1144 bytes} [Thu Jul 28 17:46:12 2022] POST /identity/v3/users => generated 286 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 17:46:12.431939 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a73107-de93-4778-853a-b932962b8ece admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:12.443165 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a73107-de93-4778-853a-b932962b8ece admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:12.447915 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256cfd33-4718-4d5e-ae8c-0a65531f05a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:12.448533 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256cfd33-4718-4d5e-ae8c-0a65531f05a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:12.448813 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256cfd33-4718-4d5e-ae8c-0a65531f05a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:12.449086 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256cfd33-4718-4d5e-ae8c-0a65531f05a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:12.449830 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256cfd33-4718-4d5e-ae8c-0a65531f05a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:12.451467 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256cfd33-4718-4d5e-ae8c-0a65531f05a9 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:12.462625 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1051/2071] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:46:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 17:46:12.466180 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ce3f81-4acd-4eaf-a627-2723340546e5 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:12.477804 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ce3f81-4acd-4eaf-a627-2723340546e5 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:12.482853 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd15e2e-ecb4-4756-9aeb-edb881d511b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:12.484398 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd15e2e-ecb4-4756-9aeb-edb881d511b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:12.484722 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd15e2e-ecb4-4756-9aeb-edb881d511b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:12.484996 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd15e2e-ecb4-4756-9aeb-edb881d511b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:12.485871 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd15e2e-ecb4-4756-9aeb-edb881d511b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:12.488387 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd15e2e-ecb4-4756-9aeb-edb881d511b3 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:12.504443 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1021/2072] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:46:12 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:46:12.505423 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256cfd33-4718-4d5e-ae8c-0a65531f05a9 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:12.516660 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256cfd33-4718-4d5e-ae8c-0a65531f05a9 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:12.521657 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426111b2-c3fa-4c48-a7c2-c4eb63808eda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:12.522320 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426111b2-c3fa-4c48-a7c2-c4eb63808eda admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:12.522597 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426111b2-c3fa-4c48-a7c2-c4eb63808eda admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:12.522868 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426111b2-c3fa-4c48-a7c2-c4eb63808eda admin admin] PATH_INFO: `/v3/projects/e4d0b17e8c8a4a329d0cb58eec7922ce/users/b0a795de11114574ad8a56517d74c5fb/roles/11f8731756fe4caca13efa61b71b11c5` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:12.525411 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:46:12.543154 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426111b2-c3fa-4c48-a7c2-c4eb63808eda admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4d0b17e8c8a4a329d0cb58eec7922ce, user_id=b0a795de11114574ad8a56517d74c5fb, role_id=11f8731756fe4caca13efa61b71b11c5)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:12.546480 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426111b2-c3fa-4c48-a7c2-c4eb63808eda admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:12.559783 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1052/2073] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:46:12 2022] PUT /identity/v3/projects/e4d0b17e8c8a4a329d0cb58eec7922ce/users/b0a795de11114574ad8a56517d74c5fb/roles/11f8731756fe4caca13efa61b71b11c5 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 17:46:12.633785 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83271d53-b561-4ca0-87d8-9a69bcb78dc0 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:12.634338 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83271d53-b561-4ca0-87d8-9a69bcb78dc0 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:12.634338 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83271d53-b561-4ca0-87d8-9a69bcb78dc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:12.642834 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:46:12.665455 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-83271d53-b561-4ca0-87d8-9a69bcb78dc0 None None] MFA Rules not processed for user `b0a795de11114574ad8a56517d74c5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:46:12.666727 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-83271d53-b561-4ca0-87d8-9a69bcb78dc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:46:12.732239 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1022/2074] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:46:12 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:46:12.743920 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426111b2-c3fa-4c48-a7c2-c4eb63808eda admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:12.745639 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-426111b2-c3fa-4c48-a7c2-c4eb63808eda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:46:12.807417 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426111b2-c3fa-4c48-a7c2-c4eb63808eda admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:12.811188 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42b1270-13d7-4116-8a31-3db2d3318abc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96d512d0c84240918dae6c4a3e8b7b79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:12.812373 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42b1270-13d7-4116-8a31-3db2d3318abc service cinder] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:12.812659 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42b1270-13d7-4116-8a31-3db2d3318abc service cinder] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:12.812931 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42b1270-13d7-4116-8a31-3db2d3318abc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:12.822303 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42b1270-13d7-4116-8a31-3db2d3318abc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:12.823773 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42b1270-13d7-4116-8a31-3db2d3318abc service cinder] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:12.833328 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1053/2075] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:46:12 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:46:12.854961 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83271d53-b561-4ca0-87d8-9a69bcb78dc0 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:12.924544 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83271d53-b561-4ca0-87d8-9a69bcb78dc0 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:12.927048 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e289509b-79be-434e-a48a-01a746777550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:12.927690 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e289509b-79be-434e-a48a-01a746777550 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:12.927966 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e289509b-79be-434e-a48a-01a746777550 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:12.928236 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e289509b-79be-434e-a48a-01a746777550 admin admin] PATH_INFO: `/v3/users/b0a795de11114574ad8a56517d74c5fb` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:12.929436 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e289509b-79be-434e-a48a-01a746777550 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0a795de11114574ad8a56517d74c5fb)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:12.931346 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e289509b-79be-434e-a48a-01a746777550 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:12.977864 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-e289509b-79be-434e-a48a-01a746777550 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0a795de11114574ad8a56517d74c5fb'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:12.985660 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-e289509b-79be-434e-a48a-01a746777550 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0a795de11114574ad8a56517d74c5fb'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:12.996358 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-e289509b-79be-434e-a48a-01a746777550 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0a795de11114574ad8a56517d74c5fb'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:12.997270 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-e289509b-79be-434e-a48a-01a746777550 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0a795de11114574ad8a56517d74c5fb'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:12.998008 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-e289509b-79be-434e-a48a-01a746777550 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0a795de11114574ad8a56517d74c5fb'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:13.018028 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1023/2076] 10.222.0.89 () {58 vars in 1224 bytes} [Thu Jul 28 17:46:12 2022] DELETE /identity/v3/users/b0a795de11114574ad8a56517d74c5fb => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:46:13.022702 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77797be-90fd-4b87-a1bc-73095f13e147 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:13.023210 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77797be-90fd-4b87-a1bc-73095f13e147 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:13.023326 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77797be-90fd-4b87-a1bc-73095f13e147 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:13.042565 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-a77797be-90fd-4b87-a1bc-73095f13e147 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:46:13.043782 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-a77797be-90fd-4b87-a1bc-73095f13e147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:46:13.109677 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1054/2077] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:46:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:46:13.119618 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e289509b-79be-434e-a48a-01a746777550 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:13.121036 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-e289509b-79be-434e-a48a-01a746777550 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:46:13.136019 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:46:13.183543 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e289509b-79be-434e-a48a-01a746777550 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:13.187746 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79d9282-902f-4f96-bc80-e615e4520127 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76817cd541294331b0023069092e8e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:13.188384 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79d9282-902f-4f96-bc80-e615e4520127 service neutron] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:13.188668 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79d9282-902f-4f96-bc80-e615e4520127 service neutron] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:13.188943 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79d9282-902f-4f96-bc80-e615e4520127 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:13.203439 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79d9282-902f-4f96-bc80-e615e4520127 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:13.204882 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79d9282-902f-4f96-bc80-e615e4520127 service neutron] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:13.218861 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1024/2078] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:46:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:46:13.553355 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77797be-90fd-4b87-a1bc-73095f13e147 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:13.554815 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-a77797be-90fd-4b87-a1bc-73095f13e147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:46:13.569269 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77797be-90fd-4b87-a1bc-73095f13e147 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:13.574351 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45d01ff-3377-458a-a6e8-33b4c71cc03f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:13.575028 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45d01ff-3377-458a-a6e8-33b4c71cc03f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:13.575310 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45d01ff-3377-458a-a6e8-33b4c71cc03f admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:13.575677 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45d01ff-3377-458a-a6e8-33b4c71cc03f admin admin] PATH_INFO: `/v3/projects/e4d0b17e8c8a4a329d0cb58eec7922ce` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:13.583390 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45d01ff-3377-458a-a6e8-33b4c71cc03f admin admin] RBAC: Authorizing `identity:delete_project(project_id=e4d0b17e8c8a4a329d0cb58eec7922ce)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:13.585605 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45d01ff-3377-458a-a6e8-33b4c71cc03f admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:13.644064 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-b45d01ff-3377-458a-a6e8-33b4c71cc03f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e4d0b17e8c8a4a329d0cb58eec7922ce', 'request_id': 'req-b45d01ff-3377-458a-a6e8-33b4c71cc03f'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:13.650234 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-b45d01ff-3377-458a-a6e8-33b4c71cc03f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e4d0b17e8c8a4a329d0cb58eec7922ce', 'request_id': 'req-b45d01ff-3377-458a-a6e8-33b4c71cc03f'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:13.658069 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-b45d01ff-3377-458a-a6e8-33b4c71cc03f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e4d0b17e8c8a4a329d0cb58eec7922ce', 'request_id': 'req-b45d01ff-3377-458a-a6e8-33b4c71cc03f'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:13.674359 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-b45d01ff-3377-458a-a6e8-33b4c71cc03f admin admin] The token cache is being invalidate because project e4d0b17e8c8a4a329d0cb58eec7922ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80324) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 17:46:13.674652 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-b45d01ff-3377-458a-a6e8-33b4c71cc03f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:13.675294 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-b45d01ff-3377-458a-a6e8-33b4c71cc03f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:13.676873 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1055/2079] 10.222.0.89 () {58 vars in 1233 bytes} [Thu Jul 28 17:46:13 2022] DELETE /identity/v3/projects/e4d0b17e8c8a4a329d0cb58eec7922ce => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:46:13.682031 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4066b5f9-27f8-4351-8a4e-607eda43fefb None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:13.682514 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4066b5f9-27f8-4351-8a4e-607eda43fefb None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:13.682603 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4066b5f9-27f8-4351-8a4e-607eda43fefb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:13.705830 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-4066b5f9-27f8-4351-8a4e-607eda43fefb None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:46:13.707073 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-4066b5f9-27f8-4351-8a4e-607eda43fefb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:46:13.772141 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1025/2080] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:46:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:46:13.779888 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf54f50-1529-4b42-8f6b-b9853e207bf3 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:13.780737 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf54f50-1529-4b42-8f6b-b9853e207bf3 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:13.781014 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf54f50-1529-4b42-8f6b-b9853e207bf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:13.801444 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-1cf54f50-1529-4b42-8f6b-b9853e207bf3 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:46:13.802658 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-1cf54f50-1529-4b42-8f6b-b9853e207bf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:46:13.810533 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1056/2081] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:46:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:46:13.815423 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4066b5f9-27f8-4351-8a4e-607eda43fefb None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:13.825125 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4066b5f9-27f8-4351-8a4e-607eda43fefb None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:13.831661 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b16c6e-0292-44dd-a11c-65217a7d195a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:13.831661 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b16c6e-0292-44dd-a11c-65217a7d195a admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:13.832000 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b16c6e-0292-44dd-a11c-65217a7d195a admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:13.832182 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b16c6e-0292-44dd-a11c-65217a7d195a admin admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:13.832815 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b16c6e-0292-44dd-a11c-65217a7d195a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:46:13.833128 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b16c6e-0292-44dd-a11c-65217a7d195a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:13.834982 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b16c6e-0292-44dd-a11c-65217a7d195a admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:13.847882 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1026/2082] 10.222.0.89 () {58 vars in 1153 bytes} [Thu Jul 28 17:46:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:46:13.852279 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf54f50-1529-4b42-8f6b-b9853e207bf3 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:13.856847 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:46:13.864301 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf54f50-1529-4b42-8f6b-b9853e207bf3 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:13.869999 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7ac0f0-e08e-4c69-9f8c-554442faf8be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:13.870606 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7ac0f0-e08e-4c69-9f8c-554442faf8be admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:13.870893 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7ac0f0-e08e-4c69-9f8c-554442faf8be admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:13.871171 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7ac0f0-e08e-4c69-9f8c-554442faf8be admin admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:13.871841 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7ac0f0-e08e-4c69-9f8c-554442faf8be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:13.874448 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7ac0f0-e08e-4c69-9f8c-554442faf8be admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:13.909401 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1057/2083] 10.222.0.89 () {60 vars in 1153 bytes} [Thu Jul 28 17:46:13 2022] POST /identity/v3/projects => generated 375 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:46:13.910536 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b16c6e-0292-44dd-a11c-65217a7d195a admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:13.921402 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b16c6e-0292-44dd-a11c-65217a7d195a admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:13.985897 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f43bee0-0bb4-4420-86ce-fe5e3e5a500c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:13.986546 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f43bee0-0bb4-4420-86ce-fe5e3e5a500c admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:13.986813 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f43bee0-0bb4-4420-86ce-fe5e3e5a500c admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:13.987088 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f43bee0-0bb4-4420-86ce-fe5e3e5a500c admin admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:13.987805 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f43bee0-0bb4-4420-86ce-fe5e3e5a500c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:13.989805 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f43bee0-0bb4-4420-86ce-fe5e3e5a500c admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:14.011575 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1027/2084] 10.222.0.89 () {60 vars in 1144 bytes} [Thu Jul 28 17:46:13 2022] POST /identity/v3/users => generated 288 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:46:14.015778 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7ac0f0-e08e-4c69-9f8c-554442faf8be admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:14.026380 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7ac0f0-e08e-4c69-9f8c-554442faf8be admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:14.031763 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ba8f5f-2982-487a-94d4-84fb6abdc1d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:14.032396 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ba8f5f-2982-487a-94d4-84fb6abdc1d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:14.032674 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ba8f5f-2982-487a-94d4-84fb6abdc1d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:14.032942 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ba8f5f-2982-487a-94d4-84fb6abdc1d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:14.033669 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ba8f5f-2982-487a-94d4-84fb6abdc1d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:14.035255 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ba8f5f-2982-487a-94d4-84fb6abdc1d2 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:14.045032 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1058/2085] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:46:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:46:14.049956 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f43bee0-0bb4-4420-86ce-fe5e3e5a500c admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:14.060340 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f43bee0-0bb4-4420-86ce-fe5e3e5a500c admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:14.065289 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbda8f5-efd4-44a0-a035-edb70ab08a4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:14.065886 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbda8f5-efd4-44a0-a035-edb70ab08a4e admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:14.066053 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbda8f5-efd4-44a0-a035-edb70ab08a4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:14.066333 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbda8f5-efd4-44a0-a035-edb70ab08a4e admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:14.067049 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbda8f5-efd4-44a0-a035-edb70ab08a4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:14.068633 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbda8f5-efd4-44a0-a035-edb70ab08a4e admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:14.079588 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1028/2086] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:46:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:46:14.083902 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ba8f5f-2982-487a-94d4-84fb6abdc1d2 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:14.094788 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ba8f5f-2982-487a-94d4-84fb6abdc1d2 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:14.099397 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395ef641-02aa-472f-b8d2-0aaef31e3882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:14.100039 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395ef641-02aa-472f-b8d2-0aaef31e3882 admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:14.100314 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395ef641-02aa-472f-b8d2-0aaef31e3882 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:14.100586 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395ef641-02aa-472f-b8d2-0aaef31e3882 admin admin] PATH_INFO: `/v3/projects/82f9067af2c94a7595aaf2fa5fea02af/users/816e5fde68904d00853b60444a1f0d18/roles/11f8731756fe4caca13efa61b71b11c5` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:14.115733 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395ef641-02aa-472f-b8d2-0aaef31e3882 admin admin] RBAC: Authorizing `identity:create_grant(project_id=82f9067af2c94a7595aaf2fa5fea02af, user_id=816e5fde68904d00853b60444a1f0d18, role_id=11f8731756fe4caca13efa61b71b11c5)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:14.118448 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395ef641-02aa-472f-b8d2-0aaef31e3882 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:14.131417 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1059/2087] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:46:14 2022] PUT /identity/v3/projects/82f9067af2c94a7595aaf2fa5fea02af/users/816e5fde68904d00853b60444a1f0d18/roles/11f8731756fe4caca13efa61b71b11c5 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:46:14.142245 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d17867-8ba5-4c66-95d9-bad284952291 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:14.142461 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d17867-8ba5-4c66-95d9-bad284952291 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:14.142735 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d17867-8ba5-4c66-95d9-bad284952291 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:14.172471 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-18d17867-8ba5-4c66-95d9-bad284952291 None None] MFA Rules not processed for user `816e5fde68904d00853b60444a1f0d18`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:46:14.174133 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-18d17867-8ba5-4c66-95d9-bad284952291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:46:14.184366 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:46:14.243466 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1029/2088] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:46:14 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:46:14.247900 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395ef641-02aa-472f-b8d2-0aaef31e3882 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:14.315797 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395ef641-02aa-472f-b8d2-0aaef31e3882 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:14.320608 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca5f90d-f5af-45b4-9008-1bc91e1afe98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:14.321286 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca5f90d-f5af-45b4-9008-1bc91e1afe98 admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:14.321576 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca5f90d-f5af-45b4-9008-1bc91e1afe98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:14.321853 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca5f90d-f5af-45b4-9008-1bc91e1afe98 admin admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:14.322540 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca5f90d-f5af-45b4-9008-1bc91e1afe98 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:14.324473 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca5f90d-f5af-45b4-9008-1bc91e1afe98 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:14.360514 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1060/2089] 10.222.0.89 () {60 vars in 1153 bytes} [Thu Jul 28 17:46:14 2022] POST /identity/v3/projects => generated 375 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:46:14.364399 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d17867-8ba5-4c66-95d9-bad284952291 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:14.383633 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d17867-8ba5-4c66-95d9-bad284952291 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:14.472565 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43d0d6d-b062-4dca-ad38-d60f9800015c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:14.473431 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43d0d6d-b062-4dca-ad38-d60f9800015c admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:14.473816 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43d0d6d-b062-4dca-ad38-d60f9800015c admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:14.474100 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43d0d6d-b062-4dca-ad38-d60f9800015c admin admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:14.474772 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43d0d6d-b062-4dca-ad38-d60f9800015c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:14.476701 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43d0d6d-b062-4dca-ad38-d60f9800015c admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:14.507018 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1030/2090] 10.222.0.89 () {60 vars in 1144 bytes} [Thu Jul 28 17:46:14 2022] POST /identity/v3/users => generated 294 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:46:14.507946 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca5f90d-f5af-45b4-9008-1bc91e1afe98 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:14.519925 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca5f90d-f5af-45b4-9008-1bc91e1afe98 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:14.524751 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a575b3e-4cd1-4a24-a752-8594e153e34e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:14.527687 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a575b3e-4cd1-4a24-a752-8594e153e34e admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:14.527687 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a575b3e-4cd1-4a24-a752-8594e153e34e admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:14.527687 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a575b3e-4cd1-4a24-a752-8594e153e34e admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:14.527687 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a575b3e-4cd1-4a24-a752-8594e153e34e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:14.528279 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a575b3e-4cd1-4a24-a752-8594e153e34e admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:14.540598 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1061/2091] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:46:14 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 17:46:14.544663 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43d0d6d-b062-4dca-ad38-d60f9800015c admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:14.554987 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43d0d6d-b062-4dca-ad38-d60f9800015c admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:14.560594 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8ea82d-82f2-455e-9622-ac950d5cf538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:14.561371 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8ea82d-82f2-455e-9622-ac950d5cf538 admin admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:14.561661 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8ea82d-82f2-455e-9622-ac950d5cf538 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:14.561935 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8ea82d-82f2-455e-9622-ac950d5cf538 admin admin] PATH_INFO: `/v3/projects/9fe059c0e7d549c386d662e27ef50ce3/users/db0ee89d805e470ebc63f662e05a184c/roles/73e457231a824ea98c28d22610be640c` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:14.580381 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8ea82d-82f2-455e-9622-ac950d5cf538 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9fe059c0e7d549c386d662e27ef50ce3, user_id=db0ee89d805e470ebc63f662e05a184c, role_id=73e457231a824ea98c28d22610be640c)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:14.583203 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8ea82d-82f2-455e-9622-ac950d5cf538 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:14.597533 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1031/2092] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:46:14 2022] PUT /identity/v3/projects/9fe059c0e7d549c386d662e27ef50ce3/users/db0ee89d805e470ebc63f662e05a184c/roles/73e457231a824ea98c28d22610be640c => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 17:46:14.601427 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a575b3e-4cd1-4a24-a752-8594e153e34e admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:14.611628 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a575b3e-4cd1-4a24-a752-8594e153e34e admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:14.670174 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e9b140-c350-476a-9dff-1db942d29aad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:14.670833 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e9b140-c350-476a-9dff-1db942d29aad admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:14.671117 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e9b140-c350-476a-9dff-1db942d29aad admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:14.671398 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e9b140-c350-476a-9dff-1db942d29aad admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:14.672097 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e9b140-c350-476a-9dff-1db942d29aad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:14.673764 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e9b140-c350-476a-9dff-1db942d29aad admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:14.683837 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1062/2093] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:46:14 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 17:46:14.688519 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8ea82d-82f2-455e-9622-ac950d5cf538 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:14.699148 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8ea82d-82f2-455e-9622-ac950d5cf538 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:14.704683 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fcc090-e6b2-416b-a886-57fe7a0acb3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:14.705379 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fcc090-e6b2-416b-a886-57fe7a0acb3e admin admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:14.705687 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fcc090-e6b2-416b-a886-57fe7a0acb3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:14.705963 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fcc090-e6b2-416b-a886-57fe7a0acb3e admin admin] PATH_INFO: `/v3/projects/9fe059c0e7d549c386d662e27ef50ce3/users/db0ee89d805e470ebc63f662e05a184c/roles/73e457231a824ea98c28d22610be640c` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:14.707915 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fcc090-e6b2-416b-a886-57fe7a0acb3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=9fe059c0e7d549c386d662e27ef50ce3, user_id=db0ee89d805e470ebc63f662e05a184c, role_id=73e457231a824ea98c28d22610be640c)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:14.710599 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fcc090-e6b2-416b-a886-57fe7a0acb3e admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:14.721318 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1032/2094] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:46:14 2022] PUT /identity/v3/projects/9fe059c0e7d549c386d662e27ef50ce3/users/db0ee89d805e470ebc63f662e05a184c/roles/73e457231a824ea98c28d22610be640c => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:46:14.732294 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48639f5f-9f42-4810-b534-ee9c4a4feb17 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:14.732585 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48639f5f-9f42-4810-b534-ee9c4a4feb17 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:14.732883 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48639f5f-9f42-4810-b534-ee9c4a4feb17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:14.761694 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-48639f5f-9f42-4810-b534-ee9c4a4feb17 None None] MFA Rules not processed for user `db0ee89d805e470ebc63f662e05a184c`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:46:14.762929 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-48639f5f-9f42-4810-b534-ee9c4a4feb17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:46:14.768062 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:46:14.842173 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1063/2095] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:46:14 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:46:14.853692 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fcc090-e6b2-416b-a886-57fe7a0acb3e admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:14.855479 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-81fcc090-e6b2-416b-a886-57fe7a0acb3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:46:14.925945 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fcc090-e6b2-416b-a886-57fe7a0acb3e admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:14.930101 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310ccb2f-57b8-4459-b9d3-8384118338e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96d512d0c84240918dae6c4a3e8b7b79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:14.930758 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310ccb2f-57b8-4459-b9d3-8384118338e0 service cinder] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:14.931042 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310ccb2f-57b8-4459-b9d3-8384118338e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:14.931317 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310ccb2f-57b8-4459-b9d3-8384118338e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:14.950716 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:46:14.993472 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310ccb2f-57b8-4459-b9d3-8384118338e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:14.994918 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310ccb2f-57b8-4459-b9d3-8384118338e0 service cinder] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:15.004603 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1033/2096] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:46:14 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:46:20.767222 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48639f5f-9f42-4810-b534-ee9c4a4feb17 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:20.833487 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48639f5f-9f42-4810-b534-ee9c4a4feb17 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:20.838906 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c825492d-fe3f-42fa-88a1-c3e89799abce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:20.839573 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c825492d-fe3f-42fa-88a1-c3e89799abce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:20.839848 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c825492d-fe3f-42fa-88a1-c3e89799abce admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:20.840119 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c825492d-fe3f-42fa-88a1-c3e89799abce admin admin] PATH_INFO: `/v3/users/816e5fde68904d00853b60444a1f0d18` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:20.841365 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c825492d-fe3f-42fa-88a1-c3e89799abce admin admin] RBAC: Authorizing `identity:delete_user(user_id=816e5fde68904d00853b60444a1f0d18)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:20.843768 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c825492d-fe3f-42fa-88a1-c3e89799abce admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:20.893538 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-c825492d-fe3f-42fa-88a1-c3e89799abce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '816e5fde68904d00853b60444a1f0d18'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:20.900356 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-c825492d-fe3f-42fa-88a1-c3e89799abce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '816e5fde68904d00853b60444a1f0d18'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:20.901065 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-c825492d-fe3f-42fa-88a1-c3e89799abce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '816e5fde68904d00853b60444a1f0d18'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:20.909032 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-c825492d-fe3f-42fa-88a1-c3e89799abce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '816e5fde68904d00853b60444a1f0d18'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:20.921450 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-c825492d-fe3f-42fa-88a1-c3e89799abce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '816e5fde68904d00853b60444a1f0d18'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:20.927375 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1064/2097] 10.222.0.89 () {58 vars in 1224 bytes} [Thu Jul 28 17:46:20 2022] DELETE /identity/v3/users/816e5fde68904d00853b60444a1f0d18 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:46:20.931780 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274f5507-8c2b-4b08-ada9-7686a6b3bfad None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:20.932253 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274f5507-8c2b-4b08-ada9-7686a6b3bfad None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:20.932347 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274f5507-8c2b-4b08-ada9-7686a6b3bfad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:20.953243 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-274f5507-8c2b-4b08-ada9-7686a6b3bfad None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:46:20.954606 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-274f5507-8c2b-4b08-ada9-7686a6b3bfad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:46:21.023098 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1034/2098] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:46:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:46:21.033754 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c825492d-fe3f-42fa-88a1-c3e89799abce admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:21.035392 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-c825492d-fe3f-42fa-88a1-c3e89799abce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:46:21.063445 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:46:21.138210 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c825492d-fe3f-42fa-88a1-c3e89799abce admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:21.148906 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b613179-66b7-498c-996d-64ee7fd0734b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76817cd541294331b0023069092e8e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:21.150282 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b613179-66b7-498c-996d-64ee7fd0734b service neutron] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:21.150282 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b613179-66b7-498c-996d-64ee7fd0734b service neutron] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:21.150736 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b613179-66b7-498c-996d-64ee7fd0734b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:21.167395 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b613179-66b7-498c-996d-64ee7fd0734b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:21.173291 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b613179-66b7-498c-996d-64ee7fd0734b service neutron] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:21.205315 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1065/2099] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:46:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:46:21.609774 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274f5507-8c2b-4b08-ada9-7686a6b3bfad None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:21.611979 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-274f5507-8c2b-4b08-ada9-7686a6b3bfad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:46:21.630055 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274f5507-8c2b-4b08-ada9-7686a6b3bfad None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:21.635266 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e7adbb-243b-4e42-b040-2a986f307a08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:21.636306 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e7adbb-243b-4e42-b040-2a986f307a08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:21.636941 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e7adbb-243b-4e42-b040-2a986f307a08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:21.637102 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e7adbb-243b-4e42-b040-2a986f307a08 admin admin] PATH_INFO: `/v3/projects/82f9067af2c94a7595aaf2fa5fea02af` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:21.638723 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e7adbb-243b-4e42-b040-2a986f307a08 admin admin] RBAC: Authorizing `identity:delete_project(project_id=82f9067af2c94a7595aaf2fa5fea02af)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:21.641472 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e7adbb-243b-4e42-b040-2a986f307a08 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:21.691894 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-46e7adbb-243b-4e42-b040-2a986f307a08 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82f9067af2c94a7595aaf2fa5fea02af', 'request_id': 'req-46e7adbb-243b-4e42-b040-2a986f307a08'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:21.698327 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-46e7adbb-243b-4e42-b040-2a986f307a08 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82f9067af2c94a7595aaf2fa5fea02af', 'request_id': 'req-46e7adbb-243b-4e42-b040-2a986f307a08'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:21.705530 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-46e7adbb-243b-4e42-b040-2a986f307a08 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82f9067af2c94a7595aaf2fa5fea02af', 'request_id': 'req-46e7adbb-243b-4e42-b040-2a986f307a08'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:21.720437 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-46e7adbb-243b-4e42-b040-2a986f307a08 admin admin] The token cache is being invalidate because project 82f9067af2c94a7595aaf2fa5fea02af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80325) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 17:46:21.720726 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-46e7adbb-243b-4e42-b040-2a986f307a08 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:21.721410 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-46e7adbb-243b-4e42-b040-2a986f307a08 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:21.722952 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1035/2100] 10.222.0.89 () {58 vars in 1233 bytes} [Thu Jul 28 17:46:21 2022] DELETE /identity/v3/projects/82f9067af2c94a7595aaf2fa5fea02af => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 17:46:21.726886 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b613179-66b7-498c-996d-64ee7fd0734b service neutron] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:21.738610 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b613179-66b7-498c-996d-64ee7fd0734b service neutron] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:21.813003 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0121e04-13ef-4c78-a635-901f8f78c270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:21.813003 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0121e04-13ef-4c78-a635-901f8f78c270 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:21.813003 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0121e04-13ef-4c78-a635-901f8f78c270 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:21.813003 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0121e04-13ef-4c78-a635-901f8f78c270 admin admin] PATH_INFO: `/v3/users/db0ee89d805e470ebc63f662e05a184c` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:21.813901 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0121e04-13ef-4c78-a635-901f8f78c270 admin admin] RBAC: Authorizing `identity:delete_user(user_id=db0ee89d805e470ebc63f662e05a184c)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:21.813901 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0121e04-13ef-4c78-a635-901f8f78c270 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:21.866041 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-c0121e04-13ef-4c78-a635-901f8f78c270 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db0ee89d805e470ebc63f662e05a184c'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:21.873838 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-c0121e04-13ef-4c78-a635-901f8f78c270 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db0ee89d805e470ebc63f662e05a184c'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:21.874980 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-c0121e04-13ef-4c78-a635-901f8f78c270 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db0ee89d805e470ebc63f662e05a184c'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:21.884146 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-c0121e04-13ef-4c78-a635-901f8f78c270 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db0ee89d805e470ebc63f662e05a184c'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:21.900553 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-c0121e04-13ef-4c78-a635-901f8f78c270 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db0ee89d805e470ebc63f662e05a184c'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:21.907951 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1066/2101] 10.222.0.89 () {58 vars in 1224 bytes} [Thu Jul 28 17:46:21 2022] DELETE /identity/v3/users/db0ee89d805e470ebc63f662e05a184c => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:46:22.197961 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e7adbb-243b-4e42-b040-2a986f307a08 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:22.199474 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-46e7adbb-243b-4e42-b040-2a986f307a08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:46:22.219672 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e7adbb-243b-4e42-b040-2a986f307a08 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:22.260998 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:46:22.282323 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3734a765-2a90-47dc-a7f5-c48b86b13bc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:22.283247 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3734a765-2a90-47dc-a7f5-c48b86b13bc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:22.283661 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3734a765-2a90-47dc-a7f5-c48b86b13bc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:22.284040 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3734a765-2a90-47dc-a7f5-c48b86b13bc2 admin admin] PATH_INFO: `/v3/projects/9fe059c0e7d549c386d662e27ef50ce3` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:22.285411 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3734a765-2a90-47dc-a7f5-c48b86b13bc2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9fe059c0e7d549c386d662e27ef50ce3)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:22.287820 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3734a765-2a90-47dc-a7f5-c48b86b13bc2 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:22.336115 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-3734a765-2a90-47dc-a7f5-c48b86b13bc2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9fe059c0e7d549c386d662e27ef50ce3', 'request_id': 'req-3734a765-2a90-47dc-a7f5-c48b86b13bc2'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:22.342189 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-3734a765-2a90-47dc-a7f5-c48b86b13bc2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9fe059c0e7d549c386d662e27ef50ce3', 'request_id': 'req-3734a765-2a90-47dc-a7f5-c48b86b13bc2'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:22.347842 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-3734a765-2a90-47dc-a7f5-c48b86b13bc2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9fe059c0e7d549c386d662e27ef50ce3', 'request_id': 'req-3734a765-2a90-47dc-a7f5-c48b86b13bc2'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:22.360289 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-3734a765-2a90-47dc-a7f5-c48b86b13bc2 admin admin] The token cache is being invalidate because project 9fe059c0e7d549c386d662e27ef50ce3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80325) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 17:46:22.360561 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-3734a765-2a90-47dc-a7f5-c48b86b13bc2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:22.361212 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-3734a765-2a90-47dc-a7f5-c48b86b13bc2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:22.362737 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1036/2102] 10.222.0.89 () {58 vars in 1233 bytes} [Thu Jul 28 17:46:22 2022] DELETE /identity/v3/projects/9fe059c0e7d549c386d662e27ef50ce3 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:46:26.269106 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0121e04-13ef-4c78-a635-901f8f78c270 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:26.271696 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-c0121e04-13ef-4c78-a635-901f8f78c270 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:46:26.288393 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0121e04-13ef-4c78-a635-901f8f78c270 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:26.344240 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:46:26.363624 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c268da-a9d7-45aa-9c88-ffe06babfec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:26.364373 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c268da-a9d7-45aa-9c88-ffe06babfec4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:26.364723 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c268da-a9d7-45aa-9c88-ffe06babfec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:26.365068 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c268da-a9d7-45aa-9c88-ffe06babfec4 admin admin] PATH_INFO: `/v3/users/6ed334d56f34486ebd8b52247d060856` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:26.366380 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c268da-a9d7-45aa-9c88-ffe06babfec4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ed334d56f34486ebd8b52247d060856)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:26.368392 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c268da-a9d7-45aa-9c88-ffe06babfec4 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:26.419251 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-f5c268da-a9d7-45aa-9c88-ffe06babfec4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ed334d56f34486ebd8b52247d060856'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:26.427412 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-f5c268da-a9d7-45aa-9c88-ffe06babfec4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ed334d56f34486ebd8b52247d060856'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:26.428108 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-f5c268da-a9d7-45aa-9c88-ffe06babfec4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ed334d56f34486ebd8b52247d060856'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:26.435616 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-f5c268da-a9d7-45aa-9c88-ffe06babfec4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ed334d56f34486ebd8b52247d060856'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:26.447760 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-f5c268da-a9d7-45aa-9c88-ffe06babfec4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ed334d56f34486ebd8b52247d060856'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:26.454369 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1067/2103] 10.222.0.89 () {58 vars in 1224 bytes} [Thu Jul 28 17:46:26 2022] DELETE /identity/v3/users/6ed334d56f34486ebd8b52247d060856 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:46:26.629055 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3734a765-2a90-47dc-a7f5-c48b86b13bc2 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:26.630536 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-3734a765-2a90-47dc-a7f5-c48b86b13bc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:46:26.648124 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3734a765-2a90-47dc-a7f5-c48b86b13bc2 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:26.707212 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487fb112-08c0-401c-9f1e-45277f7731cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:26.707855 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487fb112-08c0-401c-9f1e-45277f7731cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:26.708131 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487fb112-08c0-401c-9f1e-45277f7731cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:26.708398 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487fb112-08c0-401c-9f1e-45277f7731cd admin admin] PATH_INFO: `/v3/projects/6018db8e046e4945be4dd56ff8e8eeea` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:26.709580 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487fb112-08c0-401c-9f1e-45277f7731cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=6018db8e046e4945be4dd56ff8e8eeea)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:26.711473 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487fb112-08c0-401c-9f1e-45277f7731cd admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:26.758452 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-487fb112-08c0-401c-9f1e-45277f7731cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6018db8e046e4945be4dd56ff8e8eeea', 'request_id': 'req-487fb112-08c0-401c-9f1e-45277f7731cd'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:26.765478 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-487fb112-08c0-401c-9f1e-45277f7731cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6018db8e046e4945be4dd56ff8e8eeea', 'request_id': 'req-487fb112-08c0-401c-9f1e-45277f7731cd'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:26.771363 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-487fb112-08c0-401c-9f1e-45277f7731cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6018db8e046e4945be4dd56ff8e8eeea', 'request_id': 'req-487fb112-08c0-401c-9f1e-45277f7731cd'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:26.784942 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-487fb112-08c0-401c-9f1e-45277f7731cd admin admin] The token cache is being invalidate because project 6018db8e046e4945be4dd56ff8e8eeea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80325) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 17:46:26.785272 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-487fb112-08c0-401c-9f1e-45277f7731cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:26.785911 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-487fb112-08c0-401c-9f1e-45277f7731cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:26.787485 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1037/2104] 10.222.0.89 () {58 vars in 1233 bytes} [Thu Jul 28 17:46:26 2022] DELETE /identity/v3/projects/6018db8e046e4945be4dd56ff8e8eeea => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:46:26.792373 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeb9598-762b-425b-b746-fed7b198c4a4 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:26.792665 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeb9598-762b-425b-b746-fed7b198c4a4 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:26.792945 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeb9598-762b-425b-b746-fed7b198c4a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:26.814459 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-efeb9598-762b-425b-b746-fed7b198c4a4 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:46:26.815674 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-efeb9598-762b-425b-b746-fed7b198c4a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:46:26.875329 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1068/2105] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:46:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:46:26.883160 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ee3da6-9f73-48eb-8dd3-d138f6094724 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:26.883446 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ee3da6-9f73-48eb-8dd3-d138f6094724 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:26.883717 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ee3da6-9f73-48eb-8dd3-d138f6094724 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:26.893031 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:46:26.908186 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-29ee3da6-9f73-48eb-8dd3-d138f6094724 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:46:26.909458 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-29ee3da6-9f73-48eb-8dd3-d138f6094724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:46:26.917663 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1038/2106] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:46:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:46:26.922321 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efeb9598-762b-425b-b746-fed7b198c4a4 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:26.932030 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efeb9598-762b-425b-b746-fed7b198c4a4 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:26.937933 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a2f111-ed4c-4c4d-8bb3-23a6fa5bfcdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:26.938538 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a2f111-ed4c-4c4d-8bb3-23a6fa5bfcdc admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:26.938815 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a2f111-ed4c-4c4d-8bb3-23a6fa5bfcdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:26.939085 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a2f111-ed4c-4c4d-8bb3-23a6fa5bfcdc admin admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:26.940056 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a2f111-ed4c-4c4d-8bb3-23a6fa5bfcdc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:46:26.940358 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a2f111-ed4c-4c4d-8bb3-23a6fa5bfcdc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:26.942160 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a2f111-ed4c-4c4d-8bb3-23a6fa5bfcdc admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:26.958460 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1069/2107] 10.222.0.89 () {58 vars in 1153 bytes} [Thu Jul 28 17:46:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:46:26.962808 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ee3da6-9f73-48eb-8dd3-d138f6094724 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:26.972179 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ee3da6-9f73-48eb-8dd3-d138f6094724 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:26.976772 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f6f3b9-5001-4ed2-8451-14bbdb84eaad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:26.977411 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f6f3b9-5001-4ed2-8451-14bbdb84eaad admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:26.977689 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f6f3b9-5001-4ed2-8451-14bbdb84eaad admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:26.977959 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f6f3b9-5001-4ed2-8451-14bbdb84eaad admin admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:26.978631 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f6f3b9-5001-4ed2-8451-14bbdb84eaad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:26.980499 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f6f3b9-5001-4ed2-8451-14bbdb84eaad admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:27.017439 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1039/2108] 10.222.0.89 () {60 vars in 1153 bytes} [Thu Jul 28 17:46:26 2022] POST /identity/v3/projects => generated 393 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 17:46:27.018526 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a2f111-ed4c-4c4d-8bb3-23a6fa5bfcdc admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:27.028654 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a2f111-ed4c-4c4d-8bb3-23a6fa5bfcdc admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:27.075969 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:46:27.098953 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fafd7f9-ce56-4f8a-9663-d4525b7f64d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:27.099640 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fafd7f9-ce56-4f8a-9663-d4525b7f64d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:27.099977 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fafd7f9-ce56-4f8a-9663-d4525b7f64d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:27.100271 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fafd7f9-ce56-4f8a-9663-d4525b7f64d1 admin admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:27.101073 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fafd7f9-ce56-4f8a-9663-d4525b7f64d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:27.103054 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fafd7f9-ce56-4f8a-9663-d4525b7f64d1 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:27.132904 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1070/2109] 10.222.0.89 () {60 vars in 1144 bytes} [Thu Jul 28 17:46:27 2022] POST /identity/v3/users => generated 297 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 17:46:27.136169 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f6f3b9-5001-4ed2-8451-14bbdb84eaad admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:27.147087 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f6f3b9-5001-4ed2-8451-14bbdb84eaad admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:27.153362 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff14543-441b-4b35-9036-f460d8c04640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:27.153362 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff14543-441b-4b35-9036-f460d8c04640 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:27.153362 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff14543-441b-4b35-9036-f460d8c04640 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:27.153362 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff14543-441b-4b35-9036-f460d8c04640 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:27.153889 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff14543-441b-4b35-9036-f460d8c04640 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:27.155535 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff14543-441b-4b35-9036-f460d8c04640 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:27.166309 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1040/2110] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:46:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:46:27.170355 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fafd7f9-ce56-4f8a-9663-d4525b7f64d1 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:27.188916 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fafd7f9-ce56-4f8a-9663-d4525b7f64d1 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:27.196488 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbdd0b0-8b38-42f9-9e17-a6a85bb8a376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:27.197347 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbdd0b0-8b38-42f9-9e17-a6a85bb8a376 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:27.197769 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbdd0b0-8b38-42f9-9e17-a6a85bb8a376 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:27.198068 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbdd0b0-8b38-42f9-9e17-a6a85bb8a376 admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:27.198769 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbdd0b0-8b38-42f9-9e17-a6a85bb8a376 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:27.200651 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbdd0b0-8b38-42f9-9e17-a6a85bb8a376 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:27.215590 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1071/2111] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:46:27 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:46:27.219326 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff14543-441b-4b35-9036-f460d8c04640 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:27.227906 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff14543-441b-4b35-9036-f460d8c04640 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:27.232620 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f711c3a-5d2e-4b79-bcea-1721ba1e6790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:27.233835 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f711c3a-5d2e-4b79-bcea-1721ba1e6790 admin admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:27.234116 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f711c3a-5d2e-4b79-bcea-1721ba1e6790 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:27.234776 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f711c3a-5d2e-4b79-bcea-1721ba1e6790 admin admin] PATH_INFO: `/v3/projects/5c029976696a44cd86421b2004b903bc/users/d5df89cd44b646a8812eb39d26c492af/roles/11f8731756fe4caca13efa61b71b11c5` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:27.251971 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f711c3a-5d2e-4b79-bcea-1721ba1e6790 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c029976696a44cd86421b2004b903bc, user_id=d5df89cd44b646a8812eb39d26c492af, role_id=11f8731756fe4caca13efa61b71b11c5)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:27.254710 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f711c3a-5d2e-4b79-bcea-1721ba1e6790 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:27.273526 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1041/2112] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:46:27 2022] PUT /identity/v3/projects/5c029976696a44cd86421b2004b903bc/users/d5df89cd44b646a8812eb39d26c492af/roles/11f8731756fe4caca13efa61b71b11c5 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:46:27.284853 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b95945-03ef-42d6-a933-1d86505e9887 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:27.285631 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b95945-03ef-42d6-a933-1d86505e9887 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:27.285631 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b95945-03ef-42d6-a933-1d86505e9887 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:27.318904 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-b2b95945-03ef-42d6-a933-1d86505e9887 None None] MFA Rules not processed for user `d5df89cd44b646a8812eb39d26c492af`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:46:27.320116 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-b2b95945-03ef-42d6-a933-1d86505e9887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:46:27.394035 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1072/2113] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:46:27 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:46:27.398487 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f711c3a-5d2e-4b79-bcea-1721ba1e6790 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:27.473712 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f711c3a-5d2e-4b79-bcea-1721ba1e6790 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:27.478943 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9472dd63-6372-48c1-acc6-df740c3c10ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:27.479550 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9472dd63-6372-48c1-acc6-df740c3c10ce admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:27.479828 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9472dd63-6372-48c1-acc6-df740c3c10ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:27.480097 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9472dd63-6372-48c1-acc6-df740c3c10ce admin admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:27.480801 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9472dd63-6372-48c1-acc6-df740c3c10ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:27.482834 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9472dd63-6372-48c1-acc6-df740c3c10ce admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:27.517289 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1042/2114] 10.222.0.89 () {60 vars in 1153 bytes} [Thu Jul 28 17:46:27 2022] POST /identity/v3/projects => generated 389 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:46:27.521053 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b95945-03ef-42d6-a933-1d86505e9887 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:27.533091 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b95945-03ef-42d6-a933-1d86505e9887 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:27.600350 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082faf56-3fe0-4fb5-b0d8-4ed2ed0bb490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:27.600922 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082faf56-3fe0-4fb5-b0d8-4ed2ed0bb490 admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:27.601233 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082faf56-3fe0-4fb5-b0d8-4ed2ed0bb490 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:27.601599 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082faf56-3fe0-4fb5-b0d8-4ed2ed0bb490 admin admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:27.602343 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082faf56-3fe0-4fb5-b0d8-4ed2ed0bb490 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:27.604471 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082faf56-3fe0-4fb5-b0d8-4ed2ed0bb490 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:27.626765 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1073/2115] 10.222.0.89 () {60 vars in 1144 bytes} [Thu Jul 28 17:46:27 2022] POST /identity/v3/users => generated 301 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 17:46:27.630995 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9472dd63-6372-48c1-acc6-df740c3c10ce admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:27.641832 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9472dd63-6372-48c1-acc6-df740c3c10ce admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:27.646771 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1027bc62-3d1f-45fd-9e1c-cf1653a58133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:27.647380 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1027bc62-3d1f-45fd-9e1c-cf1653a58133 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:27.647668 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1027bc62-3d1f-45fd-9e1c-cf1653a58133 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:27.647946 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1027bc62-3d1f-45fd-9e1c-cf1653a58133 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:27.648642 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1027bc62-3d1f-45fd-9e1c-cf1653a58133 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:27.650274 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1027bc62-3d1f-45fd-9e1c-cf1653a58133 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:27.662107 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1043/2116] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:46:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:46:27.666278 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082faf56-3fe0-4fb5-b0d8-4ed2ed0bb490 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:27.677745 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082faf56-3fe0-4fb5-b0d8-4ed2ed0bb490 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:27.682300 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567cac86-05de-4e64-9bae-6a82f6e4a0b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:27.682944 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567cac86-05de-4e64-9bae-6a82f6e4a0b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:27.683221 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567cac86-05de-4e64-9bae-6a82f6e4a0b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:27.683493 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567cac86-05de-4e64-9bae-6a82f6e4a0b9 admin admin] PATH_INFO: `/v3/projects/50c7638c092e481bb89bcb31bbdc4873/users/6b0f3ca5c7aa4b55abc5b89a2b29acc4/roles/73e457231a824ea98c28d22610be640c` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:27.699252 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567cac86-05de-4e64-9bae-6a82f6e4a0b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=50c7638c092e481bb89bcb31bbdc4873, user_id=6b0f3ca5c7aa4b55abc5b89a2b29acc4, role_id=73e457231a824ea98c28d22610be640c)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:27.702112 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567cac86-05de-4e64-9bae-6a82f6e4a0b9 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:27.714822 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1074/2117] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:46:27 2022] PUT /identity/v3/projects/50c7638c092e481bb89bcb31bbdc4873/users/6b0f3ca5c7aa4b55abc5b89a2b29acc4/roles/73e457231a824ea98c28d22610be640c => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:46:27.719561 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1027bc62-3d1f-45fd-9e1c-cf1653a58133 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:27.729527 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1027bc62-3d1f-45fd-9e1c-cf1653a58133 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:27.785769 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096274ee-f7d4-4652-b43d-000862ddd82c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:27.786309 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096274ee-f7d4-4652-b43d-000862ddd82c admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:27.786590 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096274ee-f7d4-4652-b43d-000862ddd82c admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:27.786861 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096274ee-f7d4-4652-b43d-000862ddd82c admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:27.787544 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096274ee-f7d4-4652-b43d-000862ddd82c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:27.789126 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096274ee-f7d4-4652-b43d-000862ddd82c admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:27.799382 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1044/2118] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:46:27 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:46:27.803894 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567cac86-05de-4e64-9bae-6a82f6e4a0b9 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:27.814884 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567cac86-05de-4e64-9bae-6a82f6e4a0b9 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:27.819562 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1de9cea-833d-4edf-b731-bc81dc92720f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:27.820238 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1de9cea-833d-4edf-b731-bc81dc92720f admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:27.820519 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1de9cea-833d-4edf-b731-bc81dc92720f admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:27.820790 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1de9cea-833d-4edf-b731-bc81dc92720f admin admin] PATH_INFO: `/v3/projects/50c7638c092e481bb89bcb31bbdc4873/users/6b0f3ca5c7aa4b55abc5b89a2b29acc4/roles/73e457231a824ea98c28d22610be640c` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:27.822788 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1de9cea-833d-4edf-b731-bc81dc92720f admin admin] RBAC: Authorizing `identity:create_grant(project_id=50c7638c092e481bb89bcb31bbdc4873, user_id=6b0f3ca5c7aa4b55abc5b89a2b29acc4, role_id=73e457231a824ea98c28d22610be640c)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:27.825478 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1de9cea-833d-4edf-b731-bc81dc92720f admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:27.836269 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1075/2119] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:46:27 2022] PUT /identity/v3/projects/50c7638c092e481bb89bcb31bbdc4873/users/6b0f3ca5c7aa4b55abc5b89a2b29acc4/roles/73e457231a824ea98c28d22610be640c => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:46:27.846623 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acc8dd1-0320-4f2f-86df-b3ef4df70d53 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:27.846907 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acc8dd1-0320-4f2f-86df-b3ef4df70d53 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:27.847181 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acc8dd1-0320-4f2f-86df-b3ef4df70d53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:27.867267 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:46:27.878182 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-7acc8dd1-0320-4f2f-86df-b3ef4df70d53 None None] MFA Rules not processed for user `6b0f3ca5c7aa4b55abc5b89a2b29acc4`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:46:27.879484 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-7acc8dd1-0320-4f2f-86df-b3ef4df70d53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:46:27.948260 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1045/2120] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:46:27 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:46:27.959259 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1de9cea-833d-4edf-b731-bc81dc92720f admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:27.961199 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-a1de9cea-833d-4edf-b731-bc81dc92720f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:46:27.972192 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:46:28.022090 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1de9cea-833d-4edf-b731-bc81dc92720f admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:28.025733 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36aadd4e-e100-4043-84aa-0d001e67cc58 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96d512d0c84240918dae6c4a3e8b7b79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:28.026335 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36aadd4e-e100-4043-84aa-0d001e67cc58 service cinder] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:28.026613 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36aadd4e-e100-4043-84aa-0d001e67cc58 service cinder] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:28.026883 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36aadd4e-e100-4043-84aa-0d001e67cc58 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:28.036480 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36aadd4e-e100-4043-84aa-0d001e67cc58 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:28.037865 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36aadd4e-e100-4043-84aa-0d001e67cc58 service cinder] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:28.048847 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1076/2121] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:46:27 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:46:28.106229 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acc8dd1-0320-4f2f-86df-b3ef4df70d53 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:28.107668 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-7acc8dd1-0320-4f2f-86df-b3ef4df70d53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:46:28.120500 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acc8dd1-0320-4f2f-86df-b3ef4df70d53 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:28.124899 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7f67a1-a154-4f9d-8941-c92a51b9e45d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96d512d0c84240918dae6c4a3e8b7b79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:28.125675 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7f67a1-a154-4f9d-8941-c92a51b9e45d service cinder] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:28.125936 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7f67a1-a154-4f9d-8941-c92a51b9e45d service cinder] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:28.126206 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7f67a1-a154-4f9d-8941-c92a51b9e45d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:28.192206 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7f67a1-a154-4f9d-8941-c92a51b9e45d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:28.193699 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7f67a1-a154-4f9d-8941-c92a51b9e45d service cinder] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:28.203191 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1046/2122] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:46:28 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:46:29.690751 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36aadd4e-e100-4043-84aa-0d001e67cc58 service cinder] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:29.692281 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-36aadd4e-e100-4043-84aa-0d001e67cc58 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:46:29.758073 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36aadd4e-e100-4043-84aa-0d001e67cc58 service cinder] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:29.762883 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dbd958-ab5e-4f36-8149-8bbb30db0c8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:29.763503 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dbd958-ab5e-4f36-8149-8bbb30db0c8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:29.763778 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dbd958-ab5e-4f36-8149-8bbb30db0c8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:29.764047 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dbd958-ab5e-4f36-8149-8bbb30db0c8a admin admin] PATH_INFO: `/v3/users/52286cfb8c4641a394371115bef41532` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:29.765518 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dbd958-ab5e-4f36-8149-8bbb30db0c8a admin admin] RBAC: Authorizing `identity:delete_user(user_id=52286cfb8c4641a394371115bef41532)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:29.767415 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dbd958-ab5e-4f36-8149-8bbb30db0c8a admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:29.817328 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-d7dbd958-ab5e-4f36-8149-8bbb30db0c8a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52286cfb8c4641a394371115bef41532'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:29.824604 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-d7dbd958-ab5e-4f36-8149-8bbb30db0c8a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52286cfb8c4641a394371115bef41532'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:29.825381 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-d7dbd958-ab5e-4f36-8149-8bbb30db0c8a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52286cfb8c4641a394371115bef41532'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:29.834292 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-d7dbd958-ab5e-4f36-8149-8bbb30db0c8a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52286cfb8c4641a394371115bef41532'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:29.846979 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-d7dbd958-ab5e-4f36-8149-8bbb30db0c8a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52286cfb8c4641a394371115bef41532'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:29.852903 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1077/2123] 10.222.0.89 () {58 vars in 1224 bytes} [Thu Jul 28 17:46:29 2022] DELETE /identity/v3/users/52286cfb8c4641a394371115bef41532 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:46:29.857671 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579bcb16-e132-4e4f-8303-79f6e2596c02 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:29.858153 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579bcb16-e132-4e4f-8303-79f6e2596c02 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:29.858419 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579bcb16-e132-4e4f-8303-79f6e2596c02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:29.887993 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-579bcb16-e132-4e4f-8303-79f6e2596c02 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:46:29.889247 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-579bcb16-e132-4e4f-8303-79f6e2596c02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:46:29.952675 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1047/2124] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:46:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:46:29.969091 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dbd958-ab5e-4f36-8149-8bbb30db0c8a admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:29.970226 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-d7dbd958-ab5e-4f36-8149-8bbb30db0c8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:46:30.016266 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:46:30.046572 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dbd958-ab5e-4f36-8149-8bbb30db0c8a admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:30.051985 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b809ae-4cd3-4141-b565-d2bc2f0a1ef1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76817cd541294331b0023069092e8e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:30.055604 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b809ae-4cd3-4141-b565-d2bc2f0a1ef1 service neutron] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:30.055604 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b809ae-4cd3-4141-b565-d2bc2f0a1ef1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:30.055604 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b809ae-4cd3-4141-b565-d2bc2f0a1ef1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:30.075495 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b809ae-4cd3-4141-b565-d2bc2f0a1ef1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:30.079001 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b809ae-4cd3-4141-b565-d2bc2f0a1ef1 service neutron] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:30.093711 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1078/2125] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:46:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:46:30.519452 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579bcb16-e132-4e4f-8303-79f6e2596c02 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:30.521138 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-579bcb16-e132-4e4f-8303-79f6e2596c02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:46:30.535993 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579bcb16-e132-4e4f-8303-79f6e2596c02 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:30.541740 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585bbdc2-bc0e-49ed-b3d5-45648a6a5c7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:30.542396 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585bbdc2-bc0e-49ed-b3d5-45648a6a5c7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:30.542676 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585bbdc2-bc0e-49ed-b3d5-45648a6a5c7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:30.542952 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585bbdc2-bc0e-49ed-b3d5-45648a6a5c7d admin admin] PATH_INFO: `/v3/projects/72a2828ca9f14497842a061badda17fd` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:30.544134 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585bbdc2-bc0e-49ed-b3d5-45648a6a5c7d admin admin] RBAC: Authorizing `identity:delete_project(project_id=72a2828ca9f14497842a061badda17fd)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:30.546221 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585bbdc2-bc0e-49ed-b3d5-45648a6a5c7d admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:30.582229 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:46:30.598836 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-585bbdc2-bc0e-49ed-b3d5-45648a6a5c7d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72a2828ca9f14497842a061badda17fd', 'request_id': 'req-585bbdc2-bc0e-49ed-b3d5-45648a6a5c7d'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:30.605254 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-585bbdc2-bc0e-49ed-b3d5-45648a6a5c7d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72a2828ca9f14497842a061badda17fd', 'request_id': 'req-585bbdc2-bc0e-49ed-b3d5-45648a6a5c7d'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:30.616159 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-585bbdc2-bc0e-49ed-b3d5-45648a6a5c7d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72a2828ca9f14497842a061badda17fd', 'request_id': 'req-585bbdc2-bc0e-49ed-b3d5-45648a6a5c7d'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:30.629426 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-585bbdc2-bc0e-49ed-b3d5-45648a6a5c7d admin admin] The token cache is being invalidate because project 72a2828ca9f14497842a061badda17fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80325) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 17:46:30.629711 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-585bbdc2-bc0e-49ed-b3d5-45648a6a5c7d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:30.630331 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-585bbdc2-bc0e-49ed-b3d5-45648a6a5c7d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:30.631844 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1048/2126] 10.222.0.89 () {58 vars in 1233 bytes} [Thu Jul 28 17:46:30 2022] DELETE /identity/v3/projects/72a2828ca9f14497842a061badda17fd => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:46:30.636006 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b809ae-4cd3-4141-b565-d2bc2f0a1ef1 service neutron] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:30.647176 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b809ae-4cd3-4141-b565-d2bc2f0a1ef1 service neutron] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:30.704221 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889b8b90-0318-4f7d-9897-3933a3034598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:30.704885 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889b8b90-0318-4f7d-9897-3933a3034598 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:30.705122 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889b8b90-0318-4f7d-9897-3933a3034598 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:30.705428 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889b8b90-0318-4f7d-9897-3933a3034598 admin admin] PATH_INFO: `/v3/users/6536fa39890a48babe9df5598ddb66a2` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:30.706587 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889b8b90-0318-4f7d-9897-3933a3034598 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6536fa39890a48babe9df5598ddb66a2)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:30.708481 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889b8b90-0318-4f7d-9897-3933a3034598 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:30.754788 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-889b8b90-0318-4f7d-9897-3933a3034598 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6536fa39890a48babe9df5598ddb66a2'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:30.761570 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-889b8b90-0318-4f7d-9897-3933a3034598 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6536fa39890a48babe9df5598ddb66a2'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:30.762484 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-889b8b90-0318-4f7d-9897-3933a3034598 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6536fa39890a48babe9df5598ddb66a2'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:30.771616 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-889b8b90-0318-4f7d-9897-3933a3034598 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6536fa39890a48babe9df5598ddb66a2'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:30.783864 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-889b8b90-0318-4f7d-9897-3933a3034598 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6536fa39890a48babe9df5598ddb66a2'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:30.789622 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1079/2127] 10.222.0.89 () {58 vars in 1224 bytes} [Thu Jul 28 17:46:30 2022] DELETE /identity/v3/users/6536fa39890a48babe9df5598ddb66a2 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:46:31.075485 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585bbdc2-bc0e-49ed-b3d5-45648a6a5c7d admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:31.077133 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-585bbdc2-bc0e-49ed-b3d5-45648a6a5c7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:46:31.090545 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585bbdc2-bc0e-49ed-b3d5-45648a6a5c7d admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:31.148955 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41395ab-7e94-451b-9e35-89f2ded9df3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:31.149644 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41395ab-7e94-451b-9e35-89f2ded9df3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:31.149928 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41395ab-7e94-451b-9e35-89f2ded9df3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:31.150205 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41395ab-7e94-451b-9e35-89f2ded9df3b admin admin] PATH_INFO: `/v3/projects/8a7b4302ded54ebb85a507170e28d68a` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:31.152309 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41395ab-7e94-451b-9e35-89f2ded9df3b admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a7b4302ded54ebb85a507170e28d68a)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:31.154264 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41395ab-7e94-451b-9e35-89f2ded9df3b admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:31.201531 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-d41395ab-7e94-451b-9e35-89f2ded9df3b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a7b4302ded54ebb85a507170e28d68a', 'request_id': 'req-d41395ab-7e94-451b-9e35-89f2ded9df3b'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:31.209389 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-d41395ab-7e94-451b-9e35-89f2ded9df3b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a7b4302ded54ebb85a507170e28d68a', 'request_id': 'req-d41395ab-7e94-451b-9e35-89f2ded9df3b'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:31.214966 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-d41395ab-7e94-451b-9e35-89f2ded9df3b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a7b4302ded54ebb85a507170e28d68a', 'request_id': 'req-d41395ab-7e94-451b-9e35-89f2ded9df3b'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:31.231397 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-d41395ab-7e94-451b-9e35-89f2ded9df3b admin admin] The token cache is being invalidate because project 8a7b4302ded54ebb85a507170e28d68a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80325) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 17:46:31.231689 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-d41395ab-7e94-451b-9e35-89f2ded9df3b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:31.232331 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-d41395ab-7e94-451b-9e35-89f2ded9df3b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:31.234532 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1049/2128] 10.222.0.89 () {58 vars in 1233 bytes} [Thu Jul 28 17:46:31 2022] DELETE /identity/v3/projects/8a7b4302ded54ebb85a507170e28d68a => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:46:31.239879 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8119698-9c5a-4008-bfdc-4f7c881dccd8 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:31.240173 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8119698-9c5a-4008-bfdc-4f7c881dccd8 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:31.240453 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8119698-9c5a-4008-bfdc-4f7c881dccd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:31.260471 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-d8119698-9c5a-4008-bfdc-4f7c881dccd8 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:46:31.261957 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-d8119698-9c5a-4008-bfdc-4f7c881dccd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:46:31.309838 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:46:31.324326 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1080/2129] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:46:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:46:31.332655 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47158891-4aa3-411f-b72c-dd50516622f3 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:31.332940 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47158891-4aa3-411f-b72c-dd50516622f3 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:31.333250 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47158891-4aa3-411f-b72c-dd50516622f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:31.354266 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-47158891-4aa3-411f-b72c-dd50516622f3 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:46:31.355471 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-47158891-4aa3-411f-b72c-dd50516622f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:46:31.364830 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1050/2130] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:46:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:46:31.369042 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8119698-9c5a-4008-bfdc-4f7c881dccd8 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:31.391582 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8119698-9c5a-4008-bfdc-4f7c881dccd8 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:31.391582 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959e9e73-f260-4b45-b975-b1816277347e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:31.391582 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959e9e73-f260-4b45-b975-b1816277347e admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:31.391582 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959e9e73-f260-4b45-b975-b1816277347e admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:31.391968 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959e9e73-f260-4b45-b975-b1816277347e admin admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:31.391968 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959e9e73-f260-4b45-b975-b1816277347e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:46:31.391968 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959e9e73-f260-4b45-b975-b1816277347e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:31.391968 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959e9e73-f260-4b45-b975-b1816277347e admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:31.401044 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1081/2131] 10.222.0.89 () {58 vars in 1153 bytes} [Thu Jul 28 17:46:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:46:31.406257 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47158891-4aa3-411f-b72c-dd50516622f3 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:31.415414 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47158891-4aa3-411f-b72c-dd50516622f3 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:31.420126 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b473cf-e597-4286-93cb-f6af9c61a327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:31.421242 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b473cf-e597-4286-93cb-f6af9c61a327 admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:31.421538 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b473cf-e597-4286-93cb-f6af9c61a327 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:31.421810 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b473cf-e597-4286-93cb-f6af9c61a327 admin admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:31.422495 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b473cf-e597-4286-93cb-f6af9c61a327 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:31.424361 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b473cf-e597-4286-93cb-f6af9c61a327 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:31.452601 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1051/2132] 10.222.0.89 () {60 vars in 1153 bytes} [Thu Jul 28 17:46:31 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:46:31.456872 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959e9e73-f260-4b45-b975-b1816277347e admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:31.466915 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959e9e73-f260-4b45-b975-b1816277347e admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:31.525426 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa8e9d5-6228-40b0-a63a-484d63b03687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:31.525426 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa8e9d5-6228-40b0-a63a-484d63b03687 admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:31.525426 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa8e9d5-6228-40b0-a63a-484d63b03687 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:31.525426 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa8e9d5-6228-40b0-a63a-484d63b03687 admin admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:31.525943 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa8e9d5-6228-40b0-a63a-484d63b03687 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:31.527852 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa8e9d5-6228-40b0-a63a-484d63b03687 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:31.554031 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1082/2133] 10.222.0.89 () {60 vars in 1144 bytes} [Thu Jul 28 17:46:31 2022] POST /identity/v3/users => generated 293 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 17:46:31.560200 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b473cf-e597-4286-93cb-f6af9c61a327 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:31.573194 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b473cf-e597-4286-93cb-f6af9c61a327 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:31.581069 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e95bfb7-8a40-4cdf-9aa5-95f64ef88d43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:31.581925 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e95bfb7-8a40-4cdf-9aa5-95f64ef88d43 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:31.582388 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e95bfb7-8a40-4cdf-9aa5-95f64ef88d43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:31.582752 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e95bfb7-8a40-4cdf-9aa5-95f64ef88d43 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:31.583550 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e95bfb7-8a40-4cdf-9aa5-95f64ef88d43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:31.586057 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e95bfb7-8a40-4cdf-9aa5-95f64ef88d43 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:31.614609 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1052/2134] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:46:31 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:46:31.615964 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa8e9d5-6228-40b0-a63a-484d63b03687 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:31.627611 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa8e9d5-6228-40b0-a63a-484d63b03687 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:31.632370 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738e6098-3f36-48d3-9e58-e6aab58b0772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:31.632991 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738e6098-3f36-48d3-9e58-e6aab58b0772 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:31.633361 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738e6098-3f36-48d3-9e58-e6aab58b0772 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:31.633639 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738e6098-3f36-48d3-9e58-e6aab58b0772 admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:31.634342 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738e6098-3f36-48d3-9e58-e6aab58b0772 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:31.635957 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738e6098-3f36-48d3-9e58-e6aab58b0772 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:31.648999 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1083/2135] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:46:31 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:46:31.653324 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e95bfb7-8a40-4cdf-9aa5-95f64ef88d43 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:31.664713 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e95bfb7-8a40-4cdf-9aa5-95f64ef88d43 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:31.669796 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce38b8db-c179-402b-8d2e-fedfffa89225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:31.670500 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce38b8db-c179-402b-8d2e-fedfffa89225 admin admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:31.671478 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce38b8db-c179-402b-8d2e-fedfffa89225 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:31.672324 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce38b8db-c179-402b-8d2e-fedfffa89225 admin admin] PATH_INFO: `/v3/projects/6951930ac33a417996c9ca60c0aceff1/users/f0a8bb95a9164ba0b6b4a2424deb7a1d/roles/11f8731756fe4caca13efa61b71b11c5` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:31.675963 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:46:31.707115 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce38b8db-c179-402b-8d2e-fedfffa89225 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6951930ac33a417996c9ca60c0aceff1, user_id=f0a8bb95a9164ba0b6b4a2424deb7a1d, role_id=11f8731756fe4caca13efa61b71b11c5)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:31.709967 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce38b8db-c179-402b-8d2e-fedfffa89225 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:31.723117 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1053/2136] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:46:31 2022] PUT /identity/v3/projects/6951930ac33a417996c9ca60c0aceff1/users/f0a8bb95a9164ba0b6b4a2424deb7a1d/roles/11f8731756fe4caca13efa61b71b11c5 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:46:31.735186 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd47a30-d980-40bb-8c14-aa60c4ed0887 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:31.735802 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd47a30-d980-40bb-8c14-aa60c4ed0887 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:31.736179 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd47a30-d980-40bb-8c14-aa60c4ed0887 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:31.768808 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-edd47a30-d980-40bb-8c14-aa60c4ed0887 None None] MFA Rules not processed for user `f0a8bb95a9164ba0b6b4a2424deb7a1d`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:46:31.770058 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-edd47a30-d980-40bb-8c14-aa60c4ed0887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:46:31.791345 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce38b8db-c179-402b-8d2e-fedfffa89225 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:31.793836 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-ce38b8db-c179-402b-8d2e-fedfffa89225 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:46:31.851822 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1084/2137] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:46:31 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:46:31.856772 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd47a30-d980-40bb-8c14-aa60c4ed0887 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:31.872147 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce38b8db-c179-402b-8d2e-fedfffa89225 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:31.879199 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cc7900-8359-4319-9af2-04c057fa0b38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:31.880225 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cc7900-8359-4319-9af2-04c057fa0b38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:31.880674 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cc7900-8359-4319-9af2-04c057fa0b38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:31.881116 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cc7900-8359-4319-9af2-04c057fa0b38 admin admin] PATH_INFO: `/v3/users/d5df89cd44b646a8812eb39d26c492af` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:31.882563 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cc7900-8359-4319-9af2-04c057fa0b38 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d5df89cd44b646a8812eb39d26c492af)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:31.884646 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cc7900-8359-4319-9af2-04c057fa0b38 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:31.911228 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:46:31.927541 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd47a30-d980-40bb-8c14-aa60c4ed0887 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:31.932188 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ecac04-ef89-4615-bf88-93c08a6c5a8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:31.932960 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ecac04-ef89-4615-bf88-93c08a6c5a8d admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:31.933080 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ecac04-ef89-4615-bf88-93c08a6c5a8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:31.933359 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ecac04-ef89-4615-bf88-93c08a6c5a8d admin admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:31.934020 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-64cc7900-8359-4319-9af2-04c057fa0b38 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd5df89cd44b646a8812eb39d26c492af'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:31.934142 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ecac04-ef89-4615-bf88-93c08a6c5a8d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:31.935898 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ecac04-ef89-4615-bf88-93c08a6c5a8d admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:31.941398 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-64cc7900-8359-4319-9af2-04c057fa0b38 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd5df89cd44b646a8812eb39d26c492af'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:31.949489 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-64cc7900-8359-4319-9af2-04c057fa0b38 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd5df89cd44b646a8812eb39d26c492af'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:31.950281 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-64cc7900-8359-4319-9af2-04c057fa0b38 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd5df89cd44b646a8812eb39d26c492af'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:31.951026 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-64cc7900-8359-4319-9af2-04c057fa0b38 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd5df89cd44b646a8812eb39d26c492af'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:31.970387 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1085/2138] 10.222.0.89 () {60 vars in 1153 bytes} [Thu Jul 28 17:46:31 2022] POST /identity/v3/projects => generated 385 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:46:31.970569 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1054/2139] 10.222.0.89 () {58 vars in 1224 bytes} [Thu Jul 28 17:46:31 2022] DELETE /identity/v3/users/d5df89cd44b646a8812eb39d26c492af => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:46:31.971960 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ecac04-ef89-4615-bf88-93c08a6c5a8d admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:31.974336 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-77ecac04-ef89-4615-bf88-93c08a6c5a8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:46:31.975115 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc30a77-8285-47bd-bed6-15bbf8a16bc8 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:31.975409 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc30a77-8285-47bd-bed6-15bbf8a16bc8 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:31.975690 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc30a77-8285-47bd-bed6-15bbf8a16bc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:31.996844 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-7cc30a77-8285-47bd-bed6-15bbf8a16bc8 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:46:31.998105 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-7cc30a77-8285-47bd-bed6-15bbf8a16bc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:46:32.043561 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ecac04-ef89-4615-bf88-93c08a6c5a8d admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:32.058597 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1055/2140] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:46:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 17:46:32.068409 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc30a77-8285-47bd-bed6-15bbf8a16bc8 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:32.070101 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-7cc30a77-8285-47bd-bed6-15bbf8a16bc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:46:32.075231 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:46:32.103041 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fd339a-d2ff-4c72-8e65-6afdb6dc65f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:32.103708 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fd339a-d2ff-4c72-8e65-6afdb6dc65f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:32.104001 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fd339a-d2ff-4c72-8e65-6afdb6dc65f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:32.104277 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fd339a-d2ff-4c72-8e65-6afdb6dc65f9 admin admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:32.104946 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fd339a-d2ff-4c72-8e65-6afdb6dc65f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:32.106926 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fd339a-d2ff-4c72-8e65-6afdb6dc65f9 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:32.132410 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1086/2141] 10.222.0.89 () {60 vars in 1144 bytes} [Thu Jul 28 17:46:31 2022] POST /identity/v3/users => generated 293 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:46:32.133362 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc30a77-8285-47bd-bed6-15bbf8a16bc8 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:32.133800 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fd339a-d2ff-4c72-8e65-6afdb6dc65f9 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:32.137577 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311a8439-8921-4e22-8ae5-36542e9423db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76817cd541294331b0023069092e8e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:32.138204 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311a8439-8921-4e22-8ae5-36542e9423db service neutron] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:32.138487 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311a8439-8921-4e22-8ae5-36542e9423db service neutron] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:32.138971 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311a8439-8921-4e22-8ae5-36542e9423db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:32.143809 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fd339a-d2ff-4c72-8e65-6afdb6dc65f9 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:32.148621 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311a8439-8921-4e22-8ae5-36542e9423db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:32.149066 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bcdd77-4dc2-413f-b3e9-8750086be44b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:32.149680 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bcdd77-4dc2-413f-b3e9-8750086be44b admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:32.149966 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bcdd77-4dc2-413f-b3e9-8750086be44b admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:32.150060 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311a8439-8921-4e22-8ae5-36542e9423db service neutron] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:32.150269 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bcdd77-4dc2-413f-b3e9-8750086be44b admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:32.150955 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bcdd77-4dc2-413f-b3e9-8750086be44b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:32.152540 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bcdd77-4dc2-413f-b3e9-8750086be44b admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:32.160347 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1056/2142] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:46:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 17:46:32.165869 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1087/2143] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:46:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:46:32.168644 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311a8439-8921-4e22-8ae5-36542e9423db service neutron] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:32.178751 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311a8439-8921-4e22-8ae5-36542e9423db service neutron] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:32.183439 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1180f61-7b29-47ab-b830-4364d11c8a57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:32.184050 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1180f61-7b29-47ab-b830-4364d11c8a57 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:32.184335 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1180f61-7b29-47ab-b830-4364d11c8a57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:32.184612 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1180f61-7b29-47ab-b830-4364d11c8a57 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:32.185334 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1180f61-7b29-47ab-b830-4364d11c8a57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:32.187893 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1180f61-7b29-47ab-b830-4364d11c8a57 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:32.201012 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1057/2144] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:46:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:46:32.202291 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bcdd77-4dc2-413f-b3e9-8750086be44b admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:32.213692 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bcdd77-4dc2-413f-b3e9-8750086be44b admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:32.219408 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55875966-7f15-4c17-80f5-b7d3b207e140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:32.220086 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55875966-7f15-4c17-80f5-b7d3b207e140 admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:32.220371 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55875966-7f15-4c17-80f5-b7d3b207e140 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:32.220648 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55875966-7f15-4c17-80f5-b7d3b207e140 admin admin] PATH_INFO: `/v3/projects/9817f270a0544f0dab4ba4f61681ee8a/users/df07af8ad4c64770a75894dd152f90c3/roles/11f8731756fe4caca13efa61b71b11c5` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:32.237585 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55875966-7f15-4c17-80f5-b7d3b207e140 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9817f270a0544f0dab4ba4f61681ee8a, user_id=df07af8ad4c64770a75894dd152f90c3, role_id=11f8731756fe4caca13efa61b71b11c5)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:32.240364 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55875966-7f15-4c17-80f5-b7d3b207e140 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:32.254427 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1088/2145] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:46:32 2022] PUT /identity/v3/projects/9817f270a0544f0dab4ba4f61681ee8a/users/df07af8ad4c64770a75894dd152f90c3/roles/11f8731756fe4caca13efa61b71b11c5 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:46:32.266303 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b896f33c-fbb2-44df-8e17-aafcc9b0cb9a None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:32.266593 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b896f33c-fbb2-44df-8e17-aafcc9b0cb9a None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:32.266870 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b896f33c-fbb2-44df-8e17-aafcc9b0cb9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:32.296126 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-b896f33c-fbb2-44df-8e17-aafcc9b0cb9a None None] MFA Rules not processed for user `df07af8ad4c64770a75894dd152f90c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:46:32.297729 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-b896f33c-fbb2-44df-8e17-aafcc9b0cb9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:46:32.371815 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1058/2146] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:46:32 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:46:32.377392 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55875966-7f15-4c17-80f5-b7d3b207e140 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:32.425186 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:46:32.451296 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55875966-7f15-4c17-80f5-b7d3b207e140 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:32.458118 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b896f33c-fbb2-44df-8e17-aafcc9b0cb9a None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:32.459732 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4dd3c2-f811-4925-b035-3439df5f8994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:32.459990 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-b896f33c-fbb2-44df-8e17-aafcc9b0cb9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:46:32.460507 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4dd3c2-f811-4925-b035-3439df5f8994 admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:32.460797 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4dd3c2-f811-4925-b035-3439df5f8994 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:32.461097 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4dd3c2-f811-4925-b035-3439df5f8994 admin admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:32.461833 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4dd3c2-f811-4925-b035-3439df5f8994 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:32.464440 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4dd3c2-f811-4925-b035-3439df5f8994 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:32.477505 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b896f33c-fbb2-44df-8e17-aafcc9b0cb9a None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:32.485601 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bcfafd-7d26-4485-8c41-4a2a747e0c42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:32.486538 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bcfafd-7d26-4485-8c41-4a2a747e0c42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:32.486964 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bcfafd-7d26-4485-8c41-4a2a747e0c42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:32.487485 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bcfafd-7d26-4485-8c41-4a2a747e0c42 admin admin] PATH_INFO: `/v3/projects/5c029976696a44cd86421b2004b903bc` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:32.489386 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bcfafd-7d26-4485-8c41-4a2a747e0c42 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c029976696a44cd86421b2004b903bc)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:32.491523 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bcfafd-7d26-4485-8c41-4a2a747e0c42 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:32.509843 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1089/2147] 10.222.0.89 () {60 vars in 1153 bytes} [Thu Jul 28 17:46:32 2022] POST /identity/v3/projects => generated 383 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:46:32.509843 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4dd3c2-f811-4925-b035-3439df5f8994 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:32.556833 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-57bcfafd-7d26-4485-8c41-4a2a747e0c42 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c029976696a44cd86421b2004b903bc', 'request_id': 'req-57bcfafd-7d26-4485-8c41-4a2a747e0c42'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:32.563982 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-57bcfafd-7d26-4485-8c41-4a2a747e0c42 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c029976696a44cd86421b2004b903bc', 'request_id': 'req-57bcfafd-7d26-4485-8c41-4a2a747e0c42'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:32.581689 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-57bcfafd-7d26-4485-8c41-4a2a747e0c42 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c029976696a44cd86421b2004b903bc', 'request_id': 'req-57bcfafd-7d26-4485-8c41-4a2a747e0c42'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:32.583914 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4dd3c2-f811-4925-b035-3439df5f8994 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:32.600715 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-57bcfafd-7d26-4485-8c41-4a2a747e0c42 admin admin] The token cache is being invalidate because project 5c029976696a44cd86421b2004b903bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80325) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 17:46:32.601144 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-57bcfafd-7d26-4485-8c41-4a2a747e0c42 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:32.602210 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-57bcfafd-7d26-4485-8c41-4a2a747e0c42 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:32.607014 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1059/2148] 10.222.0.89 () {58 vars in 1233 bytes} [Thu Jul 28 17:46:32 2022] DELETE /identity/v3/projects/5c029976696a44cd86421b2004b903bc => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 17:46:32.609266 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bcfafd-7d26-4485-8c41-4a2a747e0c42 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:32.610789 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-57bcfafd-7d26-4485-8c41-4a2a747e0c42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:46:32.620351 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:46:32.664187 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21afc666-6038-42a8-b6e4-f53e8996448f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:32.664917 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21afc666-6038-42a8-b6e4-f53e8996448f admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:32.665307 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21afc666-6038-42a8-b6e4-f53e8996448f admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:32.665611 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21afc666-6038-42a8-b6e4-f53e8996448f admin admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:32.666287 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21afc666-6038-42a8-b6e4-f53e8996448f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:32.668222 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21afc666-6038-42a8-b6e4-f53e8996448f admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:32.682227 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bcfafd-7d26-4485-8c41-4a2a747e0c42 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:32.696619 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523d00d9-0d58-43d1-8114-fb6d5a5679fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:32.696619 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523d00d9-0d58-43d1-8114-fb6d5a5679fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:32.696619 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523d00d9-0d58-43d1-8114-fb6d5a5679fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:32.696619 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523d00d9-0d58-43d1-8114-fb6d5a5679fc admin admin] PATH_INFO: `/v3/users/6b0f3ca5c7aa4b55abc5b89a2b29acc4` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:32.701614 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523d00d9-0d58-43d1-8114-fb6d5a5679fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b0f3ca5c7aa4b55abc5b89a2b29acc4)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:32.701709 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1090/2149] 10.222.0.89 () {60 vars in 1144 bytes} [Thu Jul 28 17:46:32 2022] POST /identity/v3/users => generated 298 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 17:46:32.701943 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523d00d9-0d58-43d1-8114-fb6d5a5679fc admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:32.702056 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21afc666-6038-42a8-b6e4-f53e8996448f admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:32.710029 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-21afc666-6038-42a8-b6e4-f53e8996448f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:46:32.727751 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21afc666-6038-42a8-b6e4-f53e8996448f admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:32.753972 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e756ff8a-424d-48e5-9509-507cc9197fff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:32.754633 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e756ff8a-424d-48e5-9509-507cc9197fff admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:32.754887 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e756ff8a-424d-48e5-9509-507cc9197fff admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:32.755160 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e756ff8a-424d-48e5-9509-507cc9197fff admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:32.755952 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e756ff8a-424d-48e5-9509-507cc9197fff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:32.756983 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-523d00d9-0d58-43d1-8114-fb6d5a5679fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b0f3ca5c7aa4b55abc5b89a2b29acc4'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:32.757488 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e756ff8a-424d-48e5-9509-507cc9197fff admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:32.767343 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-523d00d9-0d58-43d1-8114-fb6d5a5679fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b0f3ca5c7aa4b55abc5b89a2b29acc4'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:32.769437 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1091/2150] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:46:32 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 17:46:32.773843 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e756ff8a-424d-48e5-9509-507cc9197fff admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:32.781582 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-523d00d9-0d58-43d1-8114-fb6d5a5679fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b0f3ca5c7aa4b55abc5b89a2b29acc4'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:32.782504 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-523d00d9-0d58-43d1-8114-fb6d5a5679fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b0f3ca5c7aa4b55abc5b89a2b29acc4'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:32.783622 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-523d00d9-0d58-43d1-8114-fb6d5a5679fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b0f3ca5c7aa4b55abc5b89a2b29acc4'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:32.785983 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e756ff8a-424d-48e5-9509-507cc9197fff admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:32.791863 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11b3284-cb4f-4483-9692-08a3bc185fe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:32.792545 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11b3284-cb4f-4483-9692-08a3bc185fe7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:32.792813 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11b3284-cb4f-4483-9692-08a3bc185fe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:32.793085 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11b3284-cb4f-4483-9692-08a3bc185fe7 admin admin] PATH_INFO: `/v3/projects/513df744e7c74eba8eda87ec03411263/users/17574845273a47338a0bd2f808f4f7c1/roles/73e457231a824ea98c28d22610be640c` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:32.808878 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1060/2151] 10.222.0.89 () {58 vars in 1224 bytes} [Thu Jul 28 17:46:32 2022] DELETE /identity/v3/users/6b0f3ca5c7aa4b55abc5b89a2b29acc4 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:46:32.816652 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11b3284-cb4f-4483-9692-08a3bc185fe7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=513df744e7c74eba8eda87ec03411263, user_id=17574845273a47338a0bd2f808f4f7c1, role_id=73e457231a824ea98c28d22610be640c)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:32.819406 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11b3284-cb4f-4483-9692-08a3bc185fe7 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:32.840348 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1092/2152] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:46:32 2022] PUT /identity/v3/projects/513df744e7c74eba8eda87ec03411263/users/17574845273a47338a0bd2f808f4f7c1/roles/73e457231a824ea98c28d22610be640c => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 17:46:32.845797 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523d00d9-0d58-43d1-8114-fb6d5a5679fc admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:32.847050 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-523d00d9-0d58-43d1-8114-fb6d5a5679fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:46:32.924141 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523d00d9-0d58-43d1-8114-fb6d5a5679fc admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:32.976087 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:46:33.013479 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db2f783-bb58-4f39-ac1e-0ec3d0de1d66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:33.015194 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db2f783-bb58-4f39-ac1e-0ec3d0de1d66 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:33.015695 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db2f783-bb58-4f39-ac1e-0ec3d0de1d66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:33.016075 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db2f783-bb58-4f39-ac1e-0ec3d0de1d66 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:33.016963 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db2f783-bb58-4f39-ac1e-0ec3d0de1d66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:33.018761 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db2f783-bb58-4f39-ac1e-0ec3d0de1d66 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:33.033595 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1061/2153] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:46:32 2022] GET /identity/v3/roles => generated 1420 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:46:33.042504 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11b3284-cb4f-4483-9692-08a3bc185fe7 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:33.059815 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11b3284-cb4f-4483-9692-08a3bc185fe7 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:33.077103 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a27cba-85e6-4a93-87f8-4fa8519a486e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:33.077817 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a27cba-85e6-4a93-87f8-4fa8519a486e admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:33.078093 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a27cba-85e6-4a93-87f8-4fa8519a486e admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:33.078422 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a27cba-85e6-4a93-87f8-4fa8519a486e admin admin] PATH_INFO: `/v3/projects/513df744e7c74eba8eda87ec03411263/users/17574845273a47338a0bd2f808f4f7c1/roles/73e457231a824ea98c28d22610be640c` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:33.080942 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a27cba-85e6-4a93-87f8-4fa8519a486e admin admin] RBAC: Authorizing `identity:create_grant(project_id=513df744e7c74eba8eda87ec03411263, user_id=17574845273a47338a0bd2f808f4f7c1, role_id=73e457231a824ea98c28d22610be640c)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:33.083640 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a27cba-85e6-4a93-87f8-4fa8519a486e admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:33.098623 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1093/2154] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:46:33 2022] PUT /identity/v3/projects/513df744e7c74eba8eda87ec03411263/users/17574845273a47338a0bd2f808f4f7c1/roles/73e457231a824ea98c28d22610be640c => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:46:33.109658 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928d64ec-b288-497d-942a-d5dfaad7f8a5 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:33.109993 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928d64ec-b288-497d-942a-d5dfaad7f8a5 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:33.110277 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928d64ec-b288-497d-942a-d5dfaad7f8a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:33.139843 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-928d64ec-b288-497d-942a-d5dfaad7f8a5 None None] MFA Rules not processed for user `17574845273a47338a0bd2f808f4f7c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:46:33.141183 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-928d64ec-b288-497d-942a-d5dfaad7f8a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:46:33.184746 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a27cba-85e6-4a93-87f8-4fa8519a486e admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:33.187252 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-d5a27cba-85e6-4a93-87f8-4fa8519a486e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:46:33.201180 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:46:33.226543 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1062/2155] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:46:33 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:46:33.239126 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928d64ec-b288-497d-942a-d5dfaad7f8a5 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:33.240553 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-928d64ec-b288-497d-942a-d5dfaad7f8a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:46:33.266671 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a27cba-85e6-4a93-87f8-4fa8519a486e admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:33.271473 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1815425-e25e-40e5-a61e-0064acf32d9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:33.272213 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1815425-e25e-40e5-a61e-0064acf32d9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:33.272383 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1815425-e25e-40e5-a61e-0064acf32d9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:33.272652 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1815425-e25e-40e5-a61e-0064acf32d9d admin admin] PATH_INFO: `/v3/projects/50c7638c092e481bb89bcb31bbdc4873` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:33.274402 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1815425-e25e-40e5-a61e-0064acf32d9d admin admin] RBAC: Authorizing `identity:delete_project(project_id=50c7638c092e481bb89bcb31bbdc4873)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:33.276559 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1815425-e25e-40e5-a61e-0064acf32d9d admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:33.299097 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928d64ec-b288-497d-942a-d5dfaad7f8a5 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:33.302776 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de915ec-7143-4ba8-9998-76d4ddd450dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96d512d0c84240918dae6c4a3e8b7b79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:33.303498 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de915ec-7143-4ba8-9998-76d4ddd450dd service cinder] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:33.303663 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de915ec-7143-4ba8-9998-76d4ddd450dd service cinder] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:33.303939 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de915ec-7143-4ba8-9998-76d4ddd450dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:33.315064 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de915ec-7143-4ba8-9998-76d4ddd450dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:33.316213 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de915ec-7143-4ba8-9998-76d4ddd450dd service cinder] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:33.326202 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1063/2156] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:46:33 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 17:46:33.328099 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-d1815425-e25e-40e5-a61e-0064acf32d9d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50c7638c092e481bb89bcb31bbdc4873', 'request_id': 'req-d1815425-e25e-40e5-a61e-0064acf32d9d'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:33.333846 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-d1815425-e25e-40e5-a61e-0064acf32d9d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50c7638c092e481bb89bcb31bbdc4873', 'request_id': 'req-d1815425-e25e-40e5-a61e-0064acf32d9d'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:33.340318 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-d1815425-e25e-40e5-a61e-0064acf32d9d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50c7638c092e481bb89bcb31bbdc4873', 'request_id': 'req-d1815425-e25e-40e5-a61e-0064acf32d9d'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:33.354985 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-d1815425-e25e-40e5-a61e-0064acf32d9d admin admin] The token cache is being invalidate because project 50c7638c092e481bb89bcb31bbdc4873 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80324) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 17:46:33.355280 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-d1815425-e25e-40e5-a61e-0064acf32d9d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:33.355927 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-d1815425-e25e-40e5-a61e-0064acf32d9d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:33.357609 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1094/2157] 10.222.0.89 () {58 vars in 1233 bytes} [Thu Jul 28 17:46:33 2022] DELETE /identity/v3/projects/50c7638c092e481bb89bcb31bbdc4873 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 17:46:33.363004 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b6f986-4c49-4f55-bc53-34532930a237 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:33.363415 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b6f986-4c49-4f55-bc53-34532930a237 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:33.365543 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b6f986-4c49-4f55-bc53-34532930a237 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:33.387209 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-81b6f986-4c49-4f55-bc53-34532930a237 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:46:33.388525 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-81b6f986-4c49-4f55-bc53-34532930a237 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:46:33.449188 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:46:33.460018 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1064/2158] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:46:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:46:33.468261 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4ada2f-85a7-41c6-ab3a-2ec50cc679cc None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:33.468654 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4ada2f-85a7-41c6-ab3a-2ec50cc679cc None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:33.469109 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4ada2f-85a7-41c6-ab3a-2ec50cc679cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:33.490382 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-0a4ada2f-85a7-41c6-ab3a-2ec50cc679cc None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:46:33.491680 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-0a4ada2f-85a7-41c6-ab3a-2ec50cc679cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:46:33.501108 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1095/2159] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:46:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:46:33.505186 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b6f986-4c49-4f55-bc53-34532930a237 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:33.516939 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b6f986-4c49-4f55-bc53-34532930a237 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:33.522097 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf91deb-3ce6-451a-9a1d-a649155aef11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:33.522826 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf91deb-3ce6-451a-9a1d-a649155aef11 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:33.523212 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf91deb-3ce6-451a-9a1d-a649155aef11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:33.523589 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf91deb-3ce6-451a-9a1d-a649155aef11 admin admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:33.524369 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf91deb-3ce6-451a-9a1d-a649155aef11 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:46:33.524761 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf91deb-3ce6-451a-9a1d-a649155aef11 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:33.526483 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf91deb-3ce6-451a-9a1d-a649155aef11 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:33.538850 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1065/2160] 10.222.0.89 () {58 vars in 1153 bytes} [Thu Jul 28 17:46:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:46:33.545232 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4ada2f-85a7-41c6-ab3a-2ec50cc679cc None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:33.553807 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4ada2f-85a7-41c6-ab3a-2ec50cc679cc None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:33.558794 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e80834-ecaf-4ce8-8cb7-6d7410f505c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:33.559409 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e80834-ecaf-4ce8-8cb7-6d7410f505c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:33.559693 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e80834-ecaf-4ce8-8cb7-6d7410f505c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:33.559971 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e80834-ecaf-4ce8-8cb7-6d7410f505c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:33.560676 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e80834-ecaf-4ce8-8cb7-6d7410f505c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:33.562690 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e80834-ecaf-4ce8-8cb7-6d7410f505c2 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:33.598024 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1096/2161] 10.222.0.89 () {60 vars in 1153 bytes} [Thu Jul 28 17:46:33 2022] POST /identity/v3/projects => generated 367 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 17:46:33.602283 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf91deb-3ce6-451a-9a1d-a649155aef11 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:33.613051 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf91deb-3ce6-451a-9a1d-a649155aef11 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:33.691518 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7988f059-29d2-4022-bb29-83027e27b055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:33.692184 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7988f059-29d2-4022-bb29-83027e27b055 admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:33.692460 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7988f059-29d2-4022-bb29-83027e27b055 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:33.692729 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7988f059-29d2-4022-bb29-83027e27b055 admin admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:33.693423 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7988f059-29d2-4022-bb29-83027e27b055 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:33.695315 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7988f059-29d2-4022-bb29-83027e27b055 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:33.716029 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1066/2162] 10.222.0.89 () {60 vars in 1144 bytes} [Thu Jul 28 17:46:33 2022] POST /identity/v3/users => generated 284 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:46:33.720666 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e80834-ecaf-4ce8-8cb7-6d7410f505c2 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:33.737610 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e80834-ecaf-4ce8-8cb7-6d7410f505c2 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:33.742499 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a4f6bd-e1ed-416c-ac04-5b00186b690b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:33.743138 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a4f6bd-e1ed-416c-ac04-5b00186b690b admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:33.743420 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a4f6bd-e1ed-416c-ac04-5b00186b690b admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:33.743696 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a4f6bd-e1ed-416c-ac04-5b00186b690b admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:33.744469 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a4f6bd-e1ed-416c-ac04-5b00186b690b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:33.746099 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a4f6bd-e1ed-416c-ac04-5b00186b690b admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:33.756887 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1097/2163] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:46:33 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:46:33.760927 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7988f059-29d2-4022-bb29-83027e27b055 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:33.771870 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7988f059-29d2-4022-bb29-83027e27b055 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:33.776577 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0561fb-dba1-4f7d-88eb-14cdc7dbfb58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:33.777220 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0561fb-dba1-4f7d-88eb-14cdc7dbfb58 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:33.777507 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0561fb-dba1-4f7d-88eb-14cdc7dbfb58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:33.777783 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0561fb-dba1-4f7d-88eb-14cdc7dbfb58 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:33.778486 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0561fb-dba1-4f7d-88eb-14cdc7dbfb58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:33.780175 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0561fb-dba1-4f7d-88eb-14cdc7dbfb58 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:33.791175 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1067/2164] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:46:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:46:33.796104 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a4f6bd-e1ed-416c-ac04-5b00186b690b admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:33.806910 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a4f6bd-e1ed-416c-ac04-5b00186b690b admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:33.818226 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698bd4b0-c2da-4c20-acca-82cd184b3e24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:33.819168 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698bd4b0-c2da-4c20-acca-82cd184b3e24 admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:33.819449 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698bd4b0-c2da-4c20-acca-82cd184b3e24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:33.819722 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698bd4b0-c2da-4c20-acca-82cd184b3e24 admin admin] PATH_INFO: `/v3/projects/48d9e9f54a254ff8aed4b595717c94f6/users/a43df42da2534e2288d185ed3dc3f824/roles/11f8731756fe4caca13efa61b71b11c5` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:33.836765 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698bd4b0-c2da-4c20-acca-82cd184b3e24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=48d9e9f54a254ff8aed4b595717c94f6, user_id=a43df42da2534e2288d185ed3dc3f824, role_id=11f8731756fe4caca13efa61b71b11c5)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:33.839475 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698bd4b0-c2da-4c20-acca-82cd184b3e24 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:33.853150 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1098/2165] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:46:33 2022] PUT /identity/v3/projects/48d9e9f54a254ff8aed4b595717c94f6/users/a43df42da2534e2288d185ed3dc3f824/roles/11f8731756fe4caca13efa61b71b11c5 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 17:46:33.865444 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4978f4-8152-4be9-bdd4-75d7f70fce80 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:33.865730 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4978f4-8152-4be9-bdd4-75d7f70fce80 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:33.866003 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4978f4-8152-4be9-bdd4-75d7f70fce80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:33.884240 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698bd4b0-c2da-4c20-acca-82cd184b3e24 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:33.886035 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-698bd4b0-c2da-4c20-acca-82cd184b3e24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:46:33.895307 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-3d4978f4-8152-4be9-bdd4-75d7f70fce80 None None] MFA Rules not processed for user `a43df42da2534e2288d185ed3dc3f824`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:46:33.896517 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-3d4978f4-8152-4be9-bdd4-75d7f70fce80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:46:33.929286 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:46:33.949952 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698bd4b0-c2da-4c20-acca-82cd184b3e24 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:33.954142 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c185a2-0493-4208-a592-1550a0ac9d4b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96d512d0c84240918dae6c4a3e8b7b79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:33.954777 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c185a2-0493-4208-a592-1550a0ac9d4b service cinder] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:33.955053 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c185a2-0493-4208-a592-1550a0ac9d4b service cinder] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:33.955325 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c185a2-0493-4208-a592-1550a0ac9d4b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:33.957593 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-66c185a2-0493-4208-a592-1550a0ac9d4b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:46:33.985051 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1068/2166] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:46:33 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:46:33.990042 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4978f4-8152-4be9-bdd4-75d7f70fce80 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:34.039233 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c185a2-0493-4208-a592-1550a0ac9d4b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:34.040669 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c185a2-0493-4208-a592-1550a0ac9d4b service cinder] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:34.050960 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1099/2167] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:46:33 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:46:34.056248 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:46:34.070016 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4978f4-8152-4be9-bdd4-75d7f70fce80 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:34.075341 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369a5adb-6cc5-4db1-aa4b-4fba573a6c16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:34.078261 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369a5adb-6cc5-4db1-aa4b-4fba573a6c16 admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:34.078627 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369a5adb-6cc5-4db1-aa4b-4fba573a6c16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:34.078968 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369a5adb-6cc5-4db1-aa4b-4fba573a6c16 admin admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:34.080014 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369a5adb-6cc5-4db1-aa4b-4fba573a6c16 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:34.082699 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369a5adb-6cc5-4db1-aa4b-4fba573a6c16 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:34.119565 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1069/2168] 10.222.0.89 () {60 vars in 1153 bytes} [Thu Jul 28 17:46:33 2022] POST /identity/v3/projects => generated 363 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 17:46:34.123539 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c185a2-0493-4208-a592-1550a0ac9d4b service cinder] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:34.141653 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c185a2-0493-4208-a592-1550a0ac9d4b service cinder] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:34.206805 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0639b29d-d609-4b77-b43e-dd070bf7de59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:34.207435 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0639b29d-d609-4b77-b43e-dd070bf7de59 admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:34.207721 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0639b29d-d609-4b77-b43e-dd070bf7de59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:34.207998 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0639b29d-d609-4b77-b43e-dd070bf7de59 admin admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:34.208712 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0639b29d-d609-4b77-b43e-dd070bf7de59 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:34.210686 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0639b29d-d609-4b77-b43e-dd070bf7de59 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:34.243304 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1100/2169] 10.222.0.89 () {60 vars in 1144 bytes} [Thu Jul 28 17:46:34 2022] POST /identity/v3/users => generated 288 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:46:34.244353 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369a5adb-6cc5-4db1-aa4b-4fba573a6c16 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:34.256831 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369a5adb-6cc5-4db1-aa4b-4fba573a6c16 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:34.261992 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f43a3b-bfc2-4d95-a059-cc2d8638e6aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:34.262690 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f43a3b-bfc2-4d95-a059-cc2d8638e6aa admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:34.263537 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f43a3b-bfc2-4d95-a059-cc2d8638e6aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:34.263537 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f43a3b-bfc2-4d95-a059-cc2d8638e6aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:34.264074 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f43a3b-bfc2-4d95-a059-cc2d8638e6aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:34.266604 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f43a3b-bfc2-4d95-a059-cc2d8638e6aa admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:34.278346 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1070/2170] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:46:34 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:46:34.282914 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0639b29d-d609-4b77-b43e-dd070bf7de59 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:34.294181 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0639b29d-d609-4b77-b43e-dd070bf7de59 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:34.301249 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7984979b-ea45-4860-9408-971e5f9acacd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:34.301932 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7984979b-ea45-4860-9408-971e5f9acacd admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:34.302289 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7984979b-ea45-4860-9408-971e5f9acacd admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:34.302568 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7984979b-ea45-4860-9408-971e5f9acacd admin admin] PATH_INFO: `/v3/projects/516c4b36aa3b4b87af71444f26849387/users/642630f919b648a885b1e6da72a2b5f7/roles/73e457231a824ea98c28d22610be640c` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:34.324641 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7984979b-ea45-4860-9408-971e5f9acacd admin admin] RBAC: Authorizing `identity:create_grant(project_id=516c4b36aa3b4b87af71444f26849387, user_id=642630f919b648a885b1e6da72a2b5f7, role_id=73e457231a824ea98c28d22610be640c)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:34.328242 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7984979b-ea45-4860-9408-971e5f9acacd admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:34.345204 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1101/2171] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:46:34 2022] PUT /identity/v3/projects/516c4b36aa3b4b87af71444f26849387/users/642630f919b648a885b1e6da72a2b5f7/roles/73e457231a824ea98c28d22610be640c => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:46:34.350435 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f43a3b-bfc2-4d95-a059-cc2d8638e6aa admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:34.368633 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f43a3b-bfc2-4d95-a059-cc2d8638e6aa admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:34.446033 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11102a9c-92f2-4d92-a54a-85820c310112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:34.446730 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11102a9c-92f2-4d92-a54a-85820c310112 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:34.447079 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11102a9c-92f2-4d92-a54a-85820c310112 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:34.447423 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11102a9c-92f2-4d92-a54a-85820c310112 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:34.448189 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11102a9c-92f2-4d92-a54a-85820c310112 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:34.449929 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11102a9c-92f2-4d92-a54a-85820c310112 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:34.461193 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1071/2172] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:46:34 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:46:34.465587 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7984979b-ea45-4860-9408-971e5f9acacd admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:34.476817 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7984979b-ea45-4860-9408-971e5f9acacd admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:34.481490 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5659ef5-a309-43cc-ae8b-3563ed6b55da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:34.482144 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5659ef5-a309-43cc-ae8b-3563ed6b55da admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:34.482425 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5659ef5-a309-43cc-ae8b-3563ed6b55da admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:34.482699 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5659ef5-a309-43cc-ae8b-3563ed6b55da admin admin] PATH_INFO: `/v3/projects/516c4b36aa3b4b87af71444f26849387/users/642630f919b648a885b1e6da72a2b5f7/roles/73e457231a824ea98c28d22610be640c` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:34.485261 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5659ef5-a309-43cc-ae8b-3563ed6b55da admin admin] RBAC: Authorizing `identity:create_grant(project_id=516c4b36aa3b4b87af71444f26849387, user_id=642630f919b648a885b1e6da72a2b5f7, role_id=73e457231a824ea98c28d22610be640c)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:34.487950 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5659ef5-a309-43cc-ae8b-3563ed6b55da admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:34.498176 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1102/2173] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:46:34 2022] PUT /identity/v3/projects/516c4b36aa3b4b87af71444f26849387/users/642630f919b648a885b1e6da72a2b5f7/roles/73e457231a824ea98c28d22610be640c => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 17:46:34.509121 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f8cdfb-0ea9-410e-ad45-5de9dbc0eb69 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:34.509448 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f8cdfb-0ea9-410e-ad45-5de9dbc0eb69 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:34.509749 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f8cdfb-0ea9-410e-ad45-5de9dbc0eb69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:34.537741 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-d4f8cdfb-0ea9-410e-ad45-5de9dbc0eb69 None None] MFA Rules not processed for user `642630f919b648a885b1e6da72a2b5f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:46:34.538950 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-d4f8cdfb-0ea9-410e-ad45-5de9dbc0eb69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:46:34.607799 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1072/2174] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:46:34 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:46:34.619732 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5659ef5-a309-43cc-ae8b-3563ed6b55da admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:34.621599 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-a5659ef5-a309-43cc-ae8b-3563ed6b55da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:46:34.667128 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:46:34.680598 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5659ef5-a309-43cc-ae8b-3563ed6b55da admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:34.684302 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2fc7cd-da94-4eb4-8783-0765887fcb22 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96d512d0c84240918dae6c4a3e8b7b79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:34.684907 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2fc7cd-da94-4eb4-8783-0765887fcb22 service cinder] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:34.685214 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2fc7cd-da94-4eb4-8783-0765887fcb22 service cinder] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:34.685494 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2fc7cd-da94-4eb4-8783-0765887fcb22 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:34.695145 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2fc7cd-da94-4eb4-8783-0765887fcb22 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:34.696522 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2fc7cd-da94-4eb4-8783-0765887fcb22 service cinder] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:34.706024 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1103/2175] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:46:34 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:46:34.786491 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f8cdfb-0ea9-410e-ad45-5de9dbc0eb69 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:34.797313 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f8cdfb-0ea9-410e-ad45-5de9dbc0eb69 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:34.801044 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0efff24-0963-4153-9b9c-b397735bfd91 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96d512d0c84240918dae6c4a3e8b7b79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:34.801693 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0efff24-0963-4153-9b9c-b397735bfd91 service cinder] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:34.802015 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0efff24-0963-4153-9b9c-b397735bfd91 service cinder] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:34.802285 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0efff24-0963-4153-9b9c-b397735bfd91 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:34.852841 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:46:34.866130 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0efff24-0963-4153-9b9c-b397735bfd91 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:34.867609 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0efff24-0963-4153-9b9c-b397735bfd91 service cinder] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:34.877723 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1073/2176] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:46:34 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:46:37.736963 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2fc7cd-da94-4eb4-8783-0765887fcb22 service cinder] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:37.738424 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-7c2fc7cd-da94-4eb4-8783-0765887fcb22 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:46:37.814642 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2fc7cd-da94-4eb4-8783-0765887fcb22 service cinder] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:37.819333 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99f1a93-90fc-49ba-a550-fde623f7f18a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:37.819982 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99f1a93-90fc-49ba-a550-fde623f7f18a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:37.820262 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99f1a93-90fc-49ba-a550-fde623f7f18a admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:37.820538 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99f1a93-90fc-49ba-a550-fde623f7f18a admin admin] PATH_INFO: `/v3/users/029cf21a98264a87adef34407ac0824c` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:37.821744 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99f1a93-90fc-49ba-a550-fde623f7f18a admin admin] RBAC: Authorizing `identity:delete_user(user_id=029cf21a98264a87adef34407ac0824c)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:37.826259 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99f1a93-90fc-49ba-a550-fde623f7f18a admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:37.875883 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-a99f1a93-90fc-49ba-a550-fde623f7f18a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '029cf21a98264a87adef34407ac0824c'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:37.883090 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-a99f1a93-90fc-49ba-a550-fde623f7f18a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '029cf21a98264a87adef34407ac0824c'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:37.883775 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-a99f1a93-90fc-49ba-a550-fde623f7f18a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '029cf21a98264a87adef34407ac0824c'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:37.892863 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-a99f1a93-90fc-49ba-a550-fde623f7f18a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '029cf21a98264a87adef34407ac0824c'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:37.906051 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-a99f1a93-90fc-49ba-a550-fde623f7f18a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '029cf21a98264a87adef34407ac0824c'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:37.912615 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1104/2177] 10.222.0.89 () {58 vars in 1224 bytes} [Thu Jul 28 17:46:37 2022] DELETE /identity/v3/users/029cf21a98264a87adef34407ac0824c => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:46:37.917306 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bd7d7f-29c4-4767-9683-812bd72bc513 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:37.917803 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bd7d7f-29c4-4767-9683-812bd72bc513 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:37.917896 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bd7d7f-29c4-4767-9683-812bd72bc513 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:37.937836 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-b3bd7d7f-29c4-4767-9683-812bd72bc513 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:46:37.939054 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-b3bd7d7f-29c4-4767-9683-812bd72bc513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:46:38.013785 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1074/2178] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:46:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:46:38.022129 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99f1a93-90fc-49ba-a550-fde623f7f18a admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:38.023753 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-a99f1a93-90fc-49ba-a550-fde623f7f18a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:46:38.113092 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99f1a93-90fc-49ba-a550-fde623f7f18a admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:38.117635 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6230b70b-0b95-401e-9fd1-cad5f8c95e58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76817cd541294331b0023069092e8e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:38.118836 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6230b70b-0b95-401e-9fd1-cad5f8c95e58 service neutron] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:38.119357 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6230b70b-0b95-401e-9fd1-cad5f8c95e58 service neutron] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:38.119765 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6230b70b-0b95-401e-9fd1-cad5f8c95e58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:38.132346 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6230b70b-0b95-401e-9fd1-cad5f8c95e58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:38.134147 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6230b70b-0b95-401e-9fd1-cad5f8c95e58 service neutron] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:38.149363 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1105/2179] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:46:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:46:38.240873 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bd7d7f-29c4-4767-9683-812bd72bc513 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:38.242335 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-b3bd7d7f-29c4-4767-9683-812bd72bc513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:46:38.306993 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bd7d7f-29c4-4767-9683-812bd72bc513 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:38.312463 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fb72ff-54b8-4d5f-9dd5-006c5cd24f79 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96d512d0c84240918dae6c4a3e8b7b79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:38.313243 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fb72ff-54b8-4d5f-9dd5-006c5cd24f79 service cinder] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:38.313789 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fb72ff-54b8-4d5f-9dd5-006c5cd24f79 service cinder] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:38.314167 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fb72ff-54b8-4d5f-9dd5-006c5cd24f79 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:38.316320 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-e5fb72ff-54b8-4d5f-9dd5-006c5cd24f79 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:46:38.397111 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fb72ff-54b8-4d5f-9dd5-006c5cd24f79 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:38.398818 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fb72ff-54b8-4d5f-9dd5-006c5cd24f79 service cinder] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:38.412397 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1075/2180] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:46:38 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:46:38.466220 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6230b70b-0b95-401e-9fd1-cad5f8c95e58 service neutron] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:38.468450 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-6230b70b-0b95-401e-9fd1-cad5f8c95e58 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:46:38.497958 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6230b70b-0b95-401e-9fd1-cad5f8c95e58 service neutron] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:38.506358 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2cde39-6a51-4a41-afbb-e2d25c98c917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:38.507706 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2cde39-6a51-4a41-afbb-e2d25c98c917 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:38.508050 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2cde39-6a51-4a41-afbb-e2d25c98c917 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:38.508515 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2cde39-6a51-4a41-afbb-e2d25c98c917 admin admin] PATH_INFO: `/v3/projects/c2e858d02366477ca704e5ac4dbd1f49` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:38.510686 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2cde39-6a51-4a41-afbb-e2d25c98c917 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2e858d02366477ca704e5ac4dbd1f49)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:38.512992 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2cde39-6a51-4a41-afbb-e2d25c98c917 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:38.516514 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:46:38.581304 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-cf2cde39-6a51-4a41-afbb-e2d25c98c917 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2e858d02366477ca704e5ac4dbd1f49', 'request_id': 'req-cf2cde39-6a51-4a41-afbb-e2d25c98c917'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:38.588173 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-cf2cde39-6a51-4a41-afbb-e2d25c98c917 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2e858d02366477ca704e5ac4dbd1f49', 'request_id': 'req-cf2cde39-6a51-4a41-afbb-e2d25c98c917'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:38.597399 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-cf2cde39-6a51-4a41-afbb-e2d25c98c917 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2e858d02366477ca704e5ac4dbd1f49', 'request_id': 'req-cf2cde39-6a51-4a41-afbb-e2d25c98c917'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:38.614692 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-cf2cde39-6a51-4a41-afbb-e2d25c98c917 admin admin] The token cache is being invalidate because project c2e858d02366477ca704e5ac4dbd1f49 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80324) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 17:46:38.614978 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-cf2cde39-6a51-4a41-afbb-e2d25c98c917 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:38.615992 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-cf2cde39-6a51-4a41-afbb-e2d25c98c917 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:38.617674 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1106/2181] 10.222.0.89 () {58 vars in 1233 bytes} [Thu Jul 28 17:46:38 2022] DELETE /identity/v3/projects/c2e858d02366477ca704e5ac4dbd1f49 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:46:38.622591 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96fe358-3f52-4f39-a6d8-1b82c5f155f1 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:38.622906 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96fe358-3f52-4f39-a6d8-1b82c5f155f1 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:38.623231 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96fe358-3f52-4f39-a6d8-1b82c5f155f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:38.644433 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-a96fe358-3f52-4f39-a6d8-1b82c5f155f1 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:46:38.645721 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-a96fe358-3f52-4f39-a6d8-1b82c5f155f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:46:38.687906 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:46:38.715310 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1076/2182] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:46:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:46:38.724029 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff1954c-849f-48db-88f6-438964dd4f83 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:38.724311 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff1954c-849f-48db-88f6-438964dd4f83 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:38.724590 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff1954c-849f-48db-88f6-438964dd4f83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:38.746168 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-6ff1954c-849f-48db-88f6-438964dd4f83 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:46:38.747376 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-6ff1954c-849f-48db-88f6-438964dd4f83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:46:38.756360 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1107/2183] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:46:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:46:38.760624 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96fe358-3f52-4f39-a6d8-1b82c5f155f1 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:38.771677 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96fe358-3f52-4f39-a6d8-1b82c5f155f1 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:38.776731 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b214905f-e3ab-4f7b-9e55-bec54a6975a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:38.777397 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b214905f-e3ab-4f7b-9e55-bec54a6975a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:38.777683 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b214905f-e3ab-4f7b-9e55-bec54a6975a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:38.777958 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b214905f-e3ab-4f7b-9e55-bec54a6975a2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:38.778669 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b214905f-e3ab-4f7b-9e55-bec54a6975a2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:46:38.778979 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b214905f-e3ab-4f7b-9e55-bec54a6975a2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:38.780684 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b214905f-e3ab-4f7b-9e55-bec54a6975a2 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:38.792681 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1077/2184] 10.222.0.89 () {58 vars in 1153 bytes} [Thu Jul 28 17:46:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:46:38.797126 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff1954c-849f-48db-88f6-438964dd4f83 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:38.806539 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff1954c-849f-48db-88f6-438964dd4f83 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:38.812064 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85856c89-01e1-4f9b-818c-e928548f1236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:38.812969 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85856c89-01e1-4f9b-818c-e928548f1236 admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:38.813945 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85856c89-01e1-4f9b-818c-e928548f1236 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:38.813945 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85856c89-01e1-4f9b-818c-e928548f1236 admin admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:38.814553 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85856c89-01e1-4f9b-818c-e928548f1236 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:38.816503 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85856c89-01e1-4f9b-818c-e928548f1236 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:38.851353 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1108/2185] 10.222.0.89 () {60 vars in 1153 bytes} [Thu Jul 28 17:46:38 2022] POST /identity/v3/projects => generated 375 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:46:38.855732 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b214905f-e3ab-4f7b-9e55-bec54a6975a2 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:38.880000 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b214905f-e3ab-4f7b-9e55-bec54a6975a2 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:38.957101 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2399ea-d731-4e21-a473-b7b84d3c666a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:38.958120 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2399ea-d731-4e21-a473-b7b84d3c666a admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:38.958405 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2399ea-d731-4e21-a473-b7b84d3c666a admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:38.958766 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2399ea-d731-4e21-a473-b7b84d3c666a admin admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:38.959496 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2399ea-d731-4e21-a473-b7b84d3c666a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:38.961621 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2399ea-d731-4e21-a473-b7b84d3c666a admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:38.986834 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1078/2186] 10.222.0.89 () {60 vars in 1144 bytes} [Thu Jul 28 17:46:38 2022] POST /identity/v3/users => generated 288 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:46:38.992137 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85856c89-01e1-4f9b-818c-e928548f1236 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:39.009129 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85856c89-01e1-4f9b-818c-e928548f1236 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:39.026022 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421e1cd0-aa75-497f-8663-dcf22146c385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:39.026646 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421e1cd0-aa75-497f-8663-dcf22146c385 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:39.026929 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421e1cd0-aa75-497f-8663-dcf22146c385 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:39.027202 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421e1cd0-aa75-497f-8663-dcf22146c385 admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:39.027911 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421e1cd0-aa75-497f-8663-dcf22146c385 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:39.029676 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421e1cd0-aa75-497f-8663-dcf22146c385 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:39.044244 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1109/2187] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:46:38 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:46:39.049771 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2399ea-d731-4e21-a473-b7b84d3c666a admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:39.063315 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2399ea-d731-4e21-a473-b7b84d3c666a admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:39.068186 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351b11f0-060b-4070-a3af-875878efb7a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:39.068839 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351b11f0-060b-4070-a3af-875878efb7a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:39.069256 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351b11f0-060b-4070-a3af-875878efb7a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:39.069421 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351b11f0-060b-4070-a3af-875878efb7a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:39.070125 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351b11f0-060b-4070-a3af-875878efb7a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:39.073930 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351b11f0-060b-4070-a3af-875878efb7a0 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:39.091131 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1079/2188] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:46:39 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:46:39.097865 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421e1cd0-aa75-497f-8663-dcf22146c385 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:39.111665 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421e1cd0-aa75-497f-8663-dcf22146c385 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:39.118445 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea96b2b2-cdd2-4215-968a-2aca953cdf05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:39.119280 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea96b2b2-cdd2-4215-968a-2aca953cdf05 admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:39.119658 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea96b2b2-cdd2-4215-968a-2aca953cdf05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:39.120034 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea96b2b2-cdd2-4215-968a-2aca953cdf05 admin admin] PATH_INFO: `/v3/projects/a83427bbf87f4dbe9e3533033c075c9e/users/902d3de9354b4eefb3d99b6e91432c74/roles/11f8731756fe4caca13efa61b71b11c5` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:39.138592 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea96b2b2-cdd2-4215-968a-2aca953cdf05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a83427bbf87f4dbe9e3533033c075c9e, user_id=902d3de9354b4eefb3d99b6e91432c74, role_id=11f8731756fe4caca13efa61b71b11c5)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:39.142015 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea96b2b2-cdd2-4215-968a-2aca953cdf05 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:39.163238 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1110/2189] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:46:39 2022] PUT /identity/v3/projects/a83427bbf87f4dbe9e3533033c075c9e/users/902d3de9354b4eefb3d99b6e91432c74/roles/11f8731756fe4caca13efa61b71b11c5 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:46:39.174438 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c587a0-d210-420f-8b3c-9cdf8b3a2187 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:39.174645 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c587a0-d210-420f-8b3c-9cdf8b3a2187 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:39.174916 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c587a0-d210-420f-8b3c-9cdf8b3a2187 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:39.209004 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-24c587a0-d210-420f-8b3c-9cdf8b3a2187 None None] MFA Rules not processed for user `902d3de9354b4eefb3d99b6e91432c74`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:46:39.210704 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-24c587a0-d210-420f-8b3c-9cdf8b3a2187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:46:39.292726 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1080/2190] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:46:39 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:46:39.308092 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea96b2b2-cdd2-4215-968a-2aca953cdf05 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:39.310738 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-ea96b2b2-cdd2-4215-968a-2aca953cdf05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:46:39.388627 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea96b2b2-cdd2-4215-968a-2aca953cdf05 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:39.392997 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74812ba8-0990-40f5-b5e3-81758af89223 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96d512d0c84240918dae6c4a3e8b7b79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:39.393642 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74812ba8-0990-40f5-b5e3-81758af89223 service cinder] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:39.393922 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74812ba8-0990-40f5-b5e3-81758af89223 service cinder] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:39.394194 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74812ba8-0990-40f5-b5e3-81758af89223 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:39.403420 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74812ba8-0990-40f5-b5e3-81758af89223 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:39.404912 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74812ba8-0990-40f5-b5e3-81758af89223 service cinder] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:39.420686 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1111/2191] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:46:39 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:46:39.814034 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c587a0-d210-420f-8b3c-9cdf8b3a2187 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:39.815517 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-24c587a0-d210-420f-8b3c-9cdf8b3a2187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:46:39.865108 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:46:39.891593 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c587a0-d210-420f-8b3c-9cdf8b3a2187 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:39.896097 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f1ff1e-69fc-45bf-9beb-9a091ee81e85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:39.896734 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f1ff1e-69fc-45bf-9beb-9a091ee81e85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:39.897013 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f1ff1e-69fc-45bf-9beb-9a091ee81e85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:39.897338 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f1ff1e-69fc-45bf-9beb-9a091ee81e85 admin admin] PATH_INFO: `/v3/users/a43df42da2534e2288d185ed3dc3f824` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:39.898502 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f1ff1e-69fc-45bf-9beb-9a091ee81e85 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a43df42da2534e2288d185ed3dc3f824)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:39.900416 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f1ff1e-69fc-45bf-9beb-9a091ee81e85 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:39.946365 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-b8f1ff1e-69fc-45bf-9beb-9a091ee81e85 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a43df42da2534e2288d185ed3dc3f824'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:39.953201 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-b8f1ff1e-69fc-45bf-9beb-9a091ee81e85 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a43df42da2534e2288d185ed3dc3f824'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:39.960888 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-b8f1ff1e-69fc-45bf-9beb-9a091ee81e85 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a43df42da2534e2288d185ed3dc3f824'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:39.961589 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-b8f1ff1e-69fc-45bf-9beb-9a091ee81e85 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a43df42da2534e2288d185ed3dc3f824'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:39.962208 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-b8f1ff1e-69fc-45bf-9beb-9a091ee81e85 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a43df42da2534e2288d185ed3dc3f824'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:39.979280 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1081/2192] 10.222.0.89 () {58 vars in 1224 bytes} [Thu Jul 28 17:46:39 2022] DELETE /identity/v3/users/a43df42da2534e2288d185ed3dc3f824 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:46:39.984407 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101f6cad-93bb-404c-b7c8-605706a725dc None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:39.984896 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101f6cad-93bb-404c-b7c8-605706a725dc None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:39.984988 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101f6cad-93bb-404c-b7c8-605706a725dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:40.004177 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-101f6cad-93bb-404c-b7c8-605706a725dc None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:46:40.005462 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-101f6cad-93bb-404c-b7c8-605706a725dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:46:40.029664 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:46:40.069779 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1112/2193] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:46:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:46:40.079645 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f1ff1e-69fc-45bf-9beb-9a091ee81e85 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:40.081095 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-b8f1ff1e-69fc-45bf-9beb-9a091ee81e85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:46:40.139884 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f1ff1e-69fc-45bf-9beb-9a091ee81e85 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:40.143414 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067f4c03-5224-4113-9068-4d97fd245f5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76817cd541294331b0023069092e8e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:40.144037 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067f4c03-5224-4113-9068-4d97fd245f5f service neutron] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:40.144323 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067f4c03-5224-4113-9068-4d97fd245f5f service neutron] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:40.144598 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067f4c03-5224-4113-9068-4d97fd245f5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:40.153818 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067f4c03-5224-4113-9068-4d97fd245f5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:40.155762 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067f4c03-5224-4113-9068-4d97fd245f5f service neutron] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:40.165679 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1082/2194] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:46:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:46:40.446326 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101f6cad-93bb-404c-b7c8-605706a725dc None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:40.447842 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-101f6cad-93bb-404c-b7c8-605706a725dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:46:40.460934 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101f6cad-93bb-404c-b7c8-605706a725dc None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:40.483997 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07016d05-c650-44a8-baaf-ada1c47bb7d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:40.484763 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07016d05-c650-44a8-baaf-ada1c47bb7d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:40.485148 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07016d05-c650-44a8-baaf-ada1c47bb7d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:40.485524 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07016d05-c650-44a8-baaf-ada1c47bb7d7 admin admin] PATH_INFO: `/v3/projects/48d9e9f54a254ff8aed4b595717c94f6` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:40.486831 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07016d05-c650-44a8-baaf-ada1c47bb7d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=48d9e9f54a254ff8aed4b595717c94f6)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:40.489020 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07016d05-c650-44a8-baaf-ada1c47bb7d7 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:40.551640 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-07016d05-c650-44a8-baaf-ada1c47bb7d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48d9e9f54a254ff8aed4b595717c94f6', 'request_id': 'req-07016d05-c650-44a8-baaf-ada1c47bb7d7'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:40.559014 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-07016d05-c650-44a8-baaf-ada1c47bb7d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48d9e9f54a254ff8aed4b595717c94f6', 'request_id': 'req-07016d05-c650-44a8-baaf-ada1c47bb7d7'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:40.565955 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-07016d05-c650-44a8-baaf-ada1c47bb7d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48d9e9f54a254ff8aed4b595717c94f6', 'request_id': 'req-07016d05-c650-44a8-baaf-ada1c47bb7d7'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:40.580527 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-07016d05-c650-44a8-baaf-ada1c47bb7d7 admin admin] The token cache is being invalidate because project 48d9e9f54a254ff8aed4b595717c94f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80324) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 17:46:40.580817 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-07016d05-c650-44a8-baaf-ada1c47bb7d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:40.581489 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-07016d05-c650-44a8-baaf-ada1c47bb7d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:40.582995 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1113/2195] 10.222.0.89 () {58 vars in 1233 bytes} [Thu Jul 28 17:46:40 2022] DELETE /identity/v3/projects/48d9e9f54a254ff8aed4b595717c94f6 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:46:40.586959 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067f4c03-5224-4113-9068-4d97fd245f5f service neutron] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:40.598140 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067f4c03-5224-4113-9068-4d97fd245f5f service neutron] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:40.660486 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a315aa02-37a6-44b5-aabb-7dff09f1db20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:40.661189 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a315aa02-37a6-44b5-aabb-7dff09f1db20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:40.661491 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a315aa02-37a6-44b5-aabb-7dff09f1db20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:40.661769 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a315aa02-37a6-44b5-aabb-7dff09f1db20 admin admin] PATH_INFO: `/v3/users/642630f919b648a885b1e6da72a2b5f7` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:40.663169 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a315aa02-37a6-44b5-aabb-7dff09f1db20 admin admin] RBAC: Authorizing `identity:delete_user(user_id=642630f919b648a885b1e6da72a2b5f7)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:40.665144 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a315aa02-37a6-44b5-aabb-7dff09f1db20 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:40.666783 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:46:40.716163 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-a315aa02-37a6-44b5-aabb-7dff09f1db20 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '642630f919b648a885b1e6da72a2b5f7'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:40.725527 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-a315aa02-37a6-44b5-aabb-7dff09f1db20 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '642630f919b648a885b1e6da72a2b5f7'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:40.736687 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-a315aa02-37a6-44b5-aabb-7dff09f1db20 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '642630f919b648a885b1e6da72a2b5f7'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:40.737730 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-a315aa02-37a6-44b5-aabb-7dff09f1db20 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '642630f919b648a885b1e6da72a2b5f7'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:40.738444 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-a315aa02-37a6-44b5-aabb-7dff09f1db20 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '642630f919b648a885b1e6da72a2b5f7'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:40.759512 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1083/2196] 10.222.0.89 () {58 vars in 1224 bytes} [Thu Jul 28 17:46:40 2022] DELETE /identity/v3/users/642630f919b648a885b1e6da72a2b5f7 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:46:40.785867 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07016d05-c650-44a8-baaf-ada1c47bb7d7 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:40.787329 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-07016d05-c650-44a8-baaf-ada1c47bb7d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:46:40.804762 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07016d05-c650-44a8-baaf-ada1c47bb7d7 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:40.866454 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7192b5-0c7c-4196-bd0f-78565bc507e9 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:40.866748 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7192b5-0c7c-4196-bd0f-78565bc507e9 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:40.867025 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7192b5-0c7c-4196-bd0f-78565bc507e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:40.882958 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6e9759-5bc2-4133-9767-efaa4dfe103b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:40.883660 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6e9759-5bc2-4133-9767-efaa4dfe103b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:40.883946 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6e9759-5bc2-4133-9767-efaa4dfe103b admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:40.884244 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6e9759-5bc2-4133-9767-efaa4dfe103b admin admin] PATH_INFO: `/v3/users/f0a8bb95a9164ba0b6b4a2424deb7a1d` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:40.885534 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6e9759-5bc2-4133-9767-efaa4dfe103b admin admin] RBAC: Authorizing `identity:delete_user(user_id=f0a8bb95a9164ba0b6b4a2424deb7a1d)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:40.887917 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6e9759-5bc2-4133-9767-efaa4dfe103b admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:40.889081 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-6e7192b5-0c7c-4196-bd0f-78565bc507e9 None None] MFA Rules not processed for user `902d3de9354b4eefb3d99b6e91432c74`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:46:40.890344 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-6e7192b5-0c7c-4196-bd0f-78565bc507e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:46:40.951770 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-1c6e9759-5bc2-4133-9767-efaa4dfe103b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f0a8bb95a9164ba0b6b4a2424deb7a1d'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:40.958537 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1084/2197] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:46:40 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 17:46:40.959042 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:46:40.961411 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-1c6e9759-5bc2-4133-9767-efaa4dfe103b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f0a8bb95a9164ba0b6b4a2424deb7a1d'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:40.962348 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-1c6e9759-5bc2-4133-9767-efaa4dfe103b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f0a8bb95a9164ba0b6b4a2424deb7a1d'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:40.971971 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-1c6e9759-5bc2-4133-9767-efaa4dfe103b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f0a8bb95a9164ba0b6b4a2424deb7a1d'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:40.986882 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-1c6e9759-5bc2-4133-9767-efaa4dfe103b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f0a8bb95a9164ba0b6b4a2424deb7a1d'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:40.993498 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1114/2198] 10.222.0.89 () {58 vars in 1224 bytes} [Thu Jul 28 17:46:40 2022] DELETE /identity/v3/users/f0a8bb95a9164ba0b6b4a2424deb7a1d => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 17:46:40.997967 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47da51dd-5600-4b89-959c-39441076ddd2 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:40.998837 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47da51dd-5600-4b89-959c-39441076ddd2 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:40.999365 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47da51dd-5600-4b89-959c-39441076ddd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:41.021796 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-47da51dd-5600-4b89-959c-39441076ddd2 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:46:41.023271 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-47da51dd-5600-4b89-959c-39441076ddd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:46:41.075079 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6e9759-5bc2-4133-9767-efaa4dfe103b admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:41.076773 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-1c6e9759-5bc2-4133-9767-efaa4dfe103b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:46:41.125698 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1085/2199] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:46:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:46:41.136213 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47da51dd-5600-4b89-959c-39441076ddd2 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:41.138711 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-47da51dd-5600-4b89-959c-39441076ddd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:46:41.174444 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6e9759-5bc2-4133-9767-efaa4dfe103b admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:41.177755 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:46:41.180641 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39de001f-f77f-4828-aaa7-982aa375f499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:41.181882 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39de001f-f77f-4828-aaa7-982aa375f499 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:41.181882 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39de001f-f77f-4828-aaa7-982aa375f499 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:41.181882 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39de001f-f77f-4828-aaa7-982aa375f499 admin admin] PATH_INFO: `/v3/projects/516c4b36aa3b4b87af71444f26849387` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:41.183042 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39de001f-f77f-4828-aaa7-982aa375f499 admin admin] RBAC: Authorizing `identity:delete_project(project_id=516c4b36aa3b4b87af71444f26849387)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:41.185182 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39de001f-f77f-4828-aaa7-982aa375f499 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:41.215611 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47da51dd-5600-4b89-959c-39441076ddd2 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:41.222220 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8d06e9-a725-4789-8ba0-6844f1fed342 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76817cd541294331b0023069092e8e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:41.222942 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8d06e9-a725-4789-8ba0-6844f1fed342 service neutron] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:41.223295 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8d06e9-a725-4789-8ba0-6844f1fed342 service neutron] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:41.223644 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8d06e9-a725-4789-8ba0-6844f1fed342 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:41.235919 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8d06e9-a725-4789-8ba0-6844f1fed342 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:41.237564 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8d06e9-a725-4789-8ba0-6844f1fed342 service neutron] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:41.244775 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-39de001f-f77f-4828-aaa7-982aa375f499 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '516c4b36aa3b4b87af71444f26849387', 'request_id': 'req-39de001f-f77f-4828-aaa7-982aa375f499'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:41.247991 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1086/2200] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:46:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 17:46:41.250868 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-39de001f-f77f-4828-aaa7-982aa375f499 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '516c4b36aa3b4b87af71444f26849387', 'request_id': 'req-39de001f-f77f-4828-aaa7-982aa375f499'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:41.257792 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-39de001f-f77f-4828-aaa7-982aa375f499 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '516c4b36aa3b4b87af71444f26849387', 'request_id': 'req-39de001f-f77f-4828-aaa7-982aa375f499'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:41.274594 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-39de001f-f77f-4828-aaa7-982aa375f499 admin admin] The token cache is being invalidate because project 516c4b36aa3b4b87af71444f26849387 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80324) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 17:46:41.275002 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-39de001f-f77f-4828-aaa7-982aa375f499 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:41.275775 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-39de001f-f77f-4828-aaa7-982aa375f499 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:41.277779 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1115/2201] 10.222.0.89 () {58 vars in 1233 bytes} [Thu Jul 28 17:46:41 2022] DELETE /identity/v3/projects/516c4b36aa3b4b87af71444f26849387 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:46:41.283395 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a487196-53e4-43e1-adbd-139c89d8a1a5 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:41.283707 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a487196-53e4-43e1-adbd-139c89d8a1a5 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:41.283992 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a487196-53e4-43e1-adbd-139c89d8a1a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:41.304585 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-9a487196-53e4-43e1-adbd-139c89d8a1a5 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:46:41.305845 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-9a487196-53e4-43e1-adbd-139c89d8a1a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:46:41.379970 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1087/2202] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:46:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:46:41.388992 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef38ab5-0b63-4883-a256-29570d25c0a3 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:41.388992 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef38ab5-0b63-4883-a256-29570d25c0a3 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:41.389245 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef38ab5-0b63-4883-a256-29570d25c0a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:41.417501 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-9ef38ab5-0b63-4883-a256-29570d25c0a3 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:46:41.418993 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-9ef38ab5-0b63-4883-a256-29570d25c0a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:46:41.435773 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1116/2203] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:46:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:46:41.438296 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a487196-53e4-43e1-adbd-139c89d8a1a5 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:41.461379 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a487196-53e4-43e1-adbd-139c89d8a1a5 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:41.466264 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2f223c-39ad-4cf3-8da9-18465dda1ef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:41.466892 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2f223c-39ad-4cf3-8da9-18465dda1ef3 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:41.467171 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2f223c-39ad-4cf3-8da9-18465dda1ef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:41.467591 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2f223c-39ad-4cf3-8da9-18465dda1ef3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:41.468850 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2f223c-39ad-4cf3-8da9-18465dda1ef3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:46:41.469186 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2f223c-39ad-4cf3-8da9-18465dda1ef3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:41.470805 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2f223c-39ad-4cf3-8da9-18465dda1ef3 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:41.489430 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1088/2204] 10.222.0.89 () {58 vars in 1153 bytes} [Thu Jul 28 17:46:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:46:41.494654 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef38ab5-0b63-4883-a256-29570d25c0a3 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:41.504707 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef38ab5-0b63-4883-a256-29570d25c0a3 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:41.510300 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69f9517-1812-475b-980b-99d1819d023a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:41.510922 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69f9517-1812-475b-980b-99d1819d023a admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:41.511201 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69f9517-1812-475b-980b-99d1819d023a admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:41.511476 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69f9517-1812-475b-980b-99d1819d023a admin admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:41.512401 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69f9517-1812-475b-980b-99d1819d023a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:41.515375 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69f9517-1812-475b-980b-99d1819d023a admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:41.582057 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1117/2205] 10.222.0.89 () {60 vars in 1153 bytes} [Thu Jul 28 17:46:41 2022] POST /identity/v3/projects => generated 383 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:46:41.583120 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2f223c-39ad-4cf3-8da9-18465dda1ef3 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:41.609331 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2f223c-39ad-4cf3-8da9-18465dda1ef3 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:41.641663 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69f9517-1812-475b-980b-99d1819d023a admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:41.643402 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-c69f9517-1812-475b-980b-99d1819d023a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:46:41.691061 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:46:41.720085 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655a07b3-e72c-4418-9eb7-fe85cba7391a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:41.720807 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655a07b3-e72c-4418-9eb7-fe85cba7391a admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:41.721276 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655a07b3-e72c-4418-9eb7-fe85cba7391a admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:41.721408 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655a07b3-e72c-4418-9eb7-fe85cba7391a admin admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:41.722091 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655a07b3-e72c-4418-9eb7-fe85cba7391a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:41.724033 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655a07b3-e72c-4418-9eb7-fe85cba7391a admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:41.742894 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69f9517-1812-475b-980b-99d1819d023a admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:41.749499 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dea966-5b7b-421e-8683-7a4704085da7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:41.750260 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dea966-5b7b-421e-8683-7a4704085da7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:41.750541 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dea966-5b7b-421e-8683-7a4704085da7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:41.750826 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dea966-5b7b-421e-8683-7a4704085da7 admin admin] PATH_INFO: `/v3/projects/6951930ac33a417996c9ca60c0aceff1` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:41.752089 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dea966-5b7b-421e-8683-7a4704085da7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6951930ac33a417996c9ca60c0aceff1)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:41.754093 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dea966-5b7b-421e-8683-7a4704085da7 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:41.767392 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1089/2206] 10.222.0.89 () {60 vars in 1144 bytes} [Thu Jul 28 17:46:41 2022] POST /identity/v3/users => generated 292 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 17:46:41.771291 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655a07b3-e72c-4418-9eb7-fe85cba7391a admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:41.791931 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655a07b3-e72c-4418-9eb7-fe85cba7391a admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:41.799738 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c9ca1f-d27f-4b3d-a58e-f9982e2f7b28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:41.800470 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c9ca1f-d27f-4b3d-a58e-f9982e2f7b28 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:41.800871 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c9ca1f-d27f-4b3d-a58e-f9982e2f7b28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:41.801256 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c9ca1f-d27f-4b3d-a58e-f9982e2f7b28 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:41.802142 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c9ca1f-d27f-4b3d-a58e-f9982e2f7b28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:41.803847 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c9ca1f-d27f-4b3d-a58e-f9982e2f7b28 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:41.817340 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1090/2207] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:46:41 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:46:41.820239 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c9ca1f-d27f-4b3d-a58e-f9982e2f7b28 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:41.835734 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c9ca1f-d27f-4b3d-a58e-f9982e2f7b28 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:41.844972 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-47dea966-5b7b-421e-8683-7a4704085da7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6951930ac33a417996c9ca60c0aceff1', 'request_id': 'req-47dea966-5b7b-421e-8683-7a4704085da7'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:41.853284 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-47dea966-5b7b-421e-8683-7a4704085da7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6951930ac33a417996c9ca60c0aceff1', 'request_id': 'req-47dea966-5b7b-421e-8683-7a4704085da7'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:41.863179 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-47dea966-5b7b-421e-8683-7a4704085da7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6951930ac33a417996c9ca60c0aceff1', 'request_id': 'req-47dea966-5b7b-421e-8683-7a4704085da7'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:41.874295 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:46:41.879427 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-47dea966-5b7b-421e-8683-7a4704085da7 admin admin] The token cache is being invalidate because project 6951930ac33a417996c9ca60c0aceff1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80324) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 17:46:41.879940 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-47dea966-5b7b-421e-8683-7a4704085da7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:41.881441 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-47dea966-5b7b-421e-8683-7a4704085da7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:41.883503 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1118/2208] 10.222.0.89 () {58 vars in 1233 bytes} [Thu Jul 28 17:46:41 2022] DELETE /identity/v3/projects/6951930ac33a417996c9ca60c0aceff1 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 17:46:41.887441 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dea966-5b7b-421e-8683-7a4704085da7 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:41.888792 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-47dea966-5b7b-421e-8683-7a4704085da7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:46:41.909375 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb1ba1f-d5ab-4ee5-9657-102f0a0084e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:41.910124 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb1ba1f-d5ab-4ee5-9657-102f0a0084e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:41.910401 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb1ba1f-d5ab-4ee5-9657-102f0a0084e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:41.910673 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb1ba1f-d5ab-4ee5-9657-102f0a0084e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:41.911367 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb1ba1f-d5ab-4ee5-9657-102f0a0084e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:41.912947 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb1ba1f-d5ab-4ee5-9657-102f0a0084e2 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:41.928459 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1091/2209] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:46:41 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 17:46:41.932724 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb1ba1f-d5ab-4ee5-9657-102f0a0084e2 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:41.934761 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-abb1ba1f-d5ab-4ee5-9657-102f0a0084e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:46:41.954243 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb1ba1f-d5ab-4ee5-9657-102f0a0084e2 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:41.959516 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf426345-bc0b-4382-93c4-f9dcab3e20f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:41.960235 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf426345-bc0b-4382-93c4-f9dcab3e20f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:41.960459 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf426345-bc0b-4382-93c4-f9dcab3e20f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:41.960766 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf426345-bc0b-4382-93c4-f9dcab3e20f1 admin admin] PATH_INFO: `/v3/projects/47a7b20146624d4aa25006a7b1c05b97/users/63578fdf9c8c4b509f8d586b5f4e15b3/roles/11f8731756fe4caca13efa61b71b11c5` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:41.966335 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dea966-5b7b-421e-8683-7a4704085da7 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:41.973301 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b896e1-66f8-4310-a829-65d1330cda67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:41.974212 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b896e1-66f8-4310-a829-65d1330cda67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:41.974719 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b896e1-66f8-4310-a829-65d1330cda67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:41.975081 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b896e1-66f8-4310-a829-65d1330cda67 admin admin] PATH_INFO: `/v3/users/df07af8ad4c64770a75894dd152f90c3` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:41.976392 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b896e1-66f8-4310-a829-65d1330cda67 admin admin] RBAC: Authorizing `identity:delete_user(user_id=df07af8ad4c64770a75894dd152f90c3)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:41.978511 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b896e1-66f8-4310-a829-65d1330cda67 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:41.980813 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf426345-bc0b-4382-93c4-f9dcab3e20f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=47a7b20146624d4aa25006a7b1c05b97, user_id=63578fdf9c8c4b509f8d586b5f4e15b3, role_id=11f8731756fe4caca13efa61b71b11c5)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:41.984335 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf426345-bc0b-4382-93c4-f9dcab3e20f1 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:42.000306 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1092/2210] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:46:41 2022] PUT /identity/v3/projects/47a7b20146624d4aa25006a7b1c05b97/users/63578fdf9c8c4b509f8d586b5f4e15b3/roles/11f8731756fe4caca13efa61b71b11c5 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 17:46:42.013131 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dccc5d-4ebb-4e8e-a5c8-f20f1b1f1462 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:42.013456 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dccc5d-4ebb-4e8e-a5c8-f20f1b1f1462 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:42.013733 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dccc5d-4ebb-4e8e-a5c8-f20f1b1f1462 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:42.034197 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-e3b896e1-66f8-4310-a829-65d1330cda67 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df07af8ad4c64770a75894dd152f90c3'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:42.041389 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-e3b896e1-66f8-4310-a829-65d1330cda67 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df07af8ad4c64770a75894dd152f90c3'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:42.042103 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-e3b896e1-66f8-4310-a829-65d1330cda67 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df07af8ad4c64770a75894dd152f90c3'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:42.043847 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-d0dccc5d-4ebb-4e8e-a5c8-f20f1b1f1462 None None] MFA Rules not processed for user `63578fdf9c8c4b509f8d586b5f4e15b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:46:42.045070 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-d0dccc5d-4ebb-4e8e-a5c8-f20f1b1f1462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:46:42.050819 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-e3b896e1-66f8-4310-a829-65d1330cda67 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df07af8ad4c64770a75894dd152f90c3'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:42.053097 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:46:42.065594 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-e3b896e1-66f8-4310-a829-65d1330cda67 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df07af8ad4c64770a75894dd152f90c3'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:42.072218 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1119/2211] 10.222.0.89 () {58 vars in 1224 bytes} [Thu Jul 28 17:46:41 2022] DELETE /identity/v3/users/df07af8ad4c64770a75894dd152f90c3 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 17:46:42.115073 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1093/2212] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:46:42 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:46:42.126950 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b896e1-66f8-4310-a829-65d1330cda67 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:42.129061 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-e3b896e1-66f8-4310-a829-65d1330cda67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:46:42.190180 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b896e1-66f8-4310-a829-65d1330cda67 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:42.193880 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a27217b-9143-441d-b7a0-8f02c0df8d80 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96d512d0c84240918dae6c4a3e8b7b79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:42.194497 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a27217b-9143-441d-b7a0-8f02c0df8d80 service cinder] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:42.194775 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a27217b-9143-441d-b7a0-8f02c0df8d80 service cinder] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:42.195042 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a27217b-9143-441d-b7a0-8f02c0df8d80 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:42.256742 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a27217b-9143-441d-b7a0-8f02c0df8d80 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:42.258206 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a27217b-9143-441d-b7a0-8f02c0df8d80 service cinder] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:42.267661 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1120/2213] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:46:42 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:46:42.361944 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dccc5d-4ebb-4e8e-a5c8-f20f1b1f1462 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:42.363590 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-d0dccc5d-4ebb-4e8e-a5c8-f20f1b1f1462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:46:42.439667 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dccc5d-4ebb-4e8e-a5c8-f20f1b1f1462 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:42.484646 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:46:42.505995 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2714066-2a38-4725-81a6-6a5c5eb672ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:42.506784 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2714066-2a38-4725-81a6-6a5c5eb672ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:42.507248 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2714066-2a38-4725-81a6-6a5c5eb672ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:42.507631 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2714066-2a38-4725-81a6-6a5c5eb672ee admin admin] PATH_INFO: `/v3/projects/9817f270a0544f0dab4ba4f61681ee8a` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:42.509413 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2714066-2a38-4725-81a6-6a5c5eb672ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=9817f270a0544f0dab4ba4f61681ee8a)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:42.511579 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2714066-2a38-4725-81a6-6a5c5eb672ee admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:42.570045 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-e2714066-2a38-4725-81a6-6a5c5eb672ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9817f270a0544f0dab4ba4f61681ee8a', 'request_id': 'req-e2714066-2a38-4725-81a6-6a5c5eb672ee'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:42.576367 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-e2714066-2a38-4725-81a6-6a5c5eb672ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9817f270a0544f0dab4ba4f61681ee8a', 'request_id': 'req-e2714066-2a38-4725-81a6-6a5c5eb672ee'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:42.582497 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-e2714066-2a38-4725-81a6-6a5c5eb672ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9817f270a0544f0dab4ba4f61681ee8a', 'request_id': 'req-e2714066-2a38-4725-81a6-6a5c5eb672ee'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:42.618896 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-e2714066-2a38-4725-81a6-6a5c5eb672ee admin admin] The token cache is being invalidate because project 9817f270a0544f0dab4ba4f61681ee8a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80325) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 17:46:42.619195 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-e2714066-2a38-4725-81a6-6a5c5eb672ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:42.620156 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-e2714066-2a38-4725-81a6-6a5c5eb672ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:42.623268 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1094/2214] 10.222.0.89 () {58 vars in 1233 bytes} [Thu Jul 28 17:46:42 2022] DELETE /identity/v3/projects/9817f270a0544f0dab4ba4f61681ee8a => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 17:46:42.627179 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a27217b-9143-441d-b7a0-8f02c0df8d80 service cinder] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:42.642338 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a27217b-9143-441d-b7a0-8f02c0df8d80 service cinder] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:42.724561 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc2abe1-66b0-4376-9f61-9495f266e2ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:42.726941 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc2abe1-66b0-4376-9f61-9495f266e2ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:42.727502 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc2abe1-66b0-4376-9f61-9495f266e2ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:42.728463 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc2abe1-66b0-4376-9f61-9495f266e2ae admin admin] PATH_INFO: `/v3/users/17574845273a47338a0bd2f808f4f7c1` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:42.730159 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc2abe1-66b0-4376-9f61-9495f266e2ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=17574845273a47338a0bd2f808f4f7c1)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:42.732626 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc2abe1-66b0-4376-9f61-9495f266e2ae admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:42.799133 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:46:42.806551 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-2bc2abe1-66b0-4376-9f61-9495f266e2ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '17574845273a47338a0bd2f808f4f7c1'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:42.815074 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-2bc2abe1-66b0-4376-9f61-9495f266e2ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '17574845273a47338a0bd2f808f4f7c1'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:42.816674 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-2bc2abe1-66b0-4376-9f61-9495f266e2ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '17574845273a47338a0bd2f808f4f7c1'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:42.830642 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-2bc2abe1-66b0-4376-9f61-9495f266e2ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '17574845273a47338a0bd2f808f4f7c1'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:42.846672 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-2bc2abe1-66b0-4376-9f61-9495f266e2ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '17574845273a47338a0bd2f808f4f7c1'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:42.855289 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1121/2215] 10.222.0.89 () {58 vars in 1224 bytes} [Thu Jul 28 17:46:42 2022] DELETE /identity/v3/users/17574845273a47338a0bd2f808f4f7c1 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 17:46:43.159487 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2714066-2a38-4725-81a6-6a5c5eb672ee admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:43.161855 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-e2714066-2a38-4725-81a6-6a5c5eb672ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:46:43.177883 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2714066-2a38-4725-81a6-6a5c5eb672ee admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:43.241840 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8587b908-078a-4805-a7c9-cf9c1ba2ecfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:43.242492 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8587b908-078a-4805-a7c9-cf9c1ba2ecfe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:43.242770 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8587b908-078a-4805-a7c9-cf9c1ba2ecfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:43.243043 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8587b908-078a-4805-a7c9-cf9c1ba2ecfe admin admin] PATH_INFO: `/v3/projects/513df744e7c74eba8eda87ec03411263` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:43.244210 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8587b908-078a-4805-a7c9-cf9c1ba2ecfe admin admin] RBAC: Authorizing `identity:delete_project(project_id=513df744e7c74eba8eda87ec03411263)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:43.246767 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8587b908-078a-4805-a7c9-cf9c1ba2ecfe admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:43.303114 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-8587b908-078a-4805-a7c9-cf9c1ba2ecfe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '513df744e7c74eba8eda87ec03411263', 'request_id': 'req-8587b908-078a-4805-a7c9-cf9c1ba2ecfe'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:43.309641 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-8587b908-078a-4805-a7c9-cf9c1ba2ecfe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '513df744e7c74eba8eda87ec03411263', 'request_id': 'req-8587b908-078a-4805-a7c9-cf9c1ba2ecfe'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:43.317196 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-8587b908-078a-4805-a7c9-cf9c1ba2ecfe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '513df744e7c74eba8eda87ec03411263', 'request_id': 'req-8587b908-078a-4805-a7c9-cf9c1ba2ecfe'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:43.345750 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-8587b908-078a-4805-a7c9-cf9c1ba2ecfe admin admin] The token cache is being invalidate because project 513df744e7c74eba8eda87ec03411263 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80325) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 17:46:43.346059 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-8587b908-078a-4805-a7c9-cf9c1ba2ecfe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:43.346829 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-8587b908-078a-4805-a7c9-cf9c1ba2ecfe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:43.348955 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1095/2216] 10.222.0.89 () {58 vars in 1233 bytes} [Thu Jul 28 17:46:43 2022] DELETE /identity/v3/projects/513df744e7c74eba8eda87ec03411263 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:46:43.355711 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e052a4e7-0268-4f47-83f6-2ea349cc928b None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:43.356455 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e052a4e7-0268-4f47-83f6-2ea349cc928b None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:43.356455 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e052a4e7-0268-4f47-83f6-2ea349cc928b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:43.377835 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-e052a4e7-0268-4f47-83f6-2ea349cc928b None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:46:43.379051 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-e052a4e7-0268-4f47-83f6-2ea349cc928b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:46:43.441391 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1122/2217] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:46:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 17:46:43.450028 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338870ff-5c5e-49c3-a4cd-2d231bf4750d None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:43.450314 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338870ff-5c5e-49c3-a4cd-2d231bf4750d None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:43.450589 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338870ff-5c5e-49c3-a4cd-2d231bf4750d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:43.459033 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:46:43.473258 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-338870ff-5c5e-49c3-a4cd-2d231bf4750d None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:46:43.474587 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-338870ff-5c5e-49c3-a4cd-2d231bf4750d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:46:43.482971 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1096/2218] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:46:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 17:46:43.488189 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e052a4e7-0268-4f47-83f6-2ea349cc928b None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:43.498066 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e052a4e7-0268-4f47-83f6-2ea349cc928b None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:43.503428 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8505b0-5ef5-46b0-9188-e44be3c7f6eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:43.504033 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8505b0-5ef5-46b0-9188-e44be3c7f6eb admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:43.504315 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8505b0-5ef5-46b0-9188-e44be3c7f6eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:43.504591 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8505b0-5ef5-46b0-9188-e44be3c7f6eb admin admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:43.505321 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8505b0-5ef5-46b0-9188-e44be3c7f6eb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:46:43.505629 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8505b0-5ef5-46b0-9188-e44be3c7f6eb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:43.507232 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8505b0-5ef5-46b0-9188-e44be3c7f6eb admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:43.524326 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1123/2219] 10.222.0.89 () {58 vars in 1153 bytes} [Thu Jul 28 17:46:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:46:43.525501 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338870ff-5c5e-49c3-a4cd-2d231bf4750d None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:43.536205 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338870ff-5c5e-49c3-a4cd-2d231bf4750d None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:43.542070 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1829b7f5-57a0-48e6-be30-5b74a3876ca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:43.542814 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1829b7f5-57a0-48e6-be30-5b74a3876ca4 admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:43.543097 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1829b7f5-57a0-48e6-be30-5b74a3876ca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:43.543543 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1829b7f5-57a0-48e6-be30-5b74a3876ca4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:43.544225 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1829b7f5-57a0-48e6-be30-5b74a3876ca4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:43.547070 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1829b7f5-57a0-48e6-be30-5b74a3876ca4 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:43.579636 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1097/2220] 10.222.0.89 () {60 vars in 1153 bytes} [Thu Jul 28 17:46:43 2022] POST /identity/v3/projects => generated 399 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:46:43.583741 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8505b0-5ef5-46b0-9188-e44be3c7f6eb admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:43.597524 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8505b0-5ef5-46b0-9188-e44be3c7f6eb admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:43.685918 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182ccaaf-1cd7-4d62-96eb-825d8434f7f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:43.686597 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182ccaaf-1cd7-4d62-96eb-825d8434f7f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:43.686877 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182ccaaf-1cd7-4d62-96eb-825d8434f7f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:43.687252 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182ccaaf-1cd7-4d62-96eb-825d8434f7f4 admin admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:43.688141 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182ccaaf-1cd7-4d62-96eb-825d8434f7f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:43.690053 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182ccaaf-1cd7-4d62-96eb-825d8434f7f4 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:43.721320 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1124/2221] 10.222.0.89 () {60 vars in 1144 bytes} [Thu Jul 28 17:46:43 2022] POST /identity/v3/users => generated 300 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 17:46:43.723340 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1829b7f5-57a0-48e6-be30-5b74a3876ca4 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:43.737545 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1829b7f5-57a0-48e6-be30-5b74a3876ca4 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:43.742909 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb0d6ad-ec4a-42e2-989f-7f1d0e6317d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:43.743536 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb0d6ad-ec4a-42e2-989f-7f1d0e6317d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:43.743821 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb0d6ad-ec4a-42e2-989f-7f1d0e6317d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:43.744201 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb0d6ad-ec4a-42e2-989f-7f1d0e6317d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:43.745387 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb0d6ad-ec4a-42e2-989f-7f1d0e6317d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:43.747609 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb0d6ad-ec4a-42e2-989f-7f1d0e6317d1 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:43.761207 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1098/2222] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:46:43 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 17:46:43.765753 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182ccaaf-1cd7-4d62-96eb-825d8434f7f4 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:43.778973 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182ccaaf-1cd7-4d62-96eb-825d8434f7f4 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:43.786536 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cc7a7b-16a8-4e07-9e2d-4529558140db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:43.787181 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cc7a7b-16a8-4e07-9e2d-4529558140db admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:43.787462 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cc7a7b-16a8-4e07-9e2d-4529558140db admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:43.788379 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cc7a7b-16a8-4e07-9e2d-4529558140db admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:43.788648 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cc7a7b-16a8-4e07-9e2d-4529558140db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:43.791312 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cc7a7b-16a8-4e07-9e2d-4529558140db admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:43.813661 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1125/2223] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:46:43 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 17:46:43.818117 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb0d6ad-ec4a-42e2-989f-7f1d0e6317d1 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:43.836829 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb0d6ad-ec4a-42e2-989f-7f1d0e6317d1 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:43.843333 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08d974a-ba7f-47e0-a9c6-12002c26318e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:43.844945 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08d974a-ba7f-47e0-a9c6-12002c26318e admin admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:43.845319 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08d974a-ba7f-47e0-a9c6-12002c26318e admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:43.845596 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08d974a-ba7f-47e0-a9c6-12002c26318e admin admin] PATH_INFO: `/v3/projects/34f32d48602e422b8d67ca1f4b2b0644/users/8a128dbe6fb245e78e5a2605ae6360e8/roles/11f8731756fe4caca13efa61b71b11c5` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:43.868232 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08d974a-ba7f-47e0-a9c6-12002c26318e admin admin] RBAC: Authorizing `identity:create_grant(project_id=34f32d48602e422b8d67ca1f4b2b0644, user_id=8a128dbe6fb245e78e5a2605ae6360e8, role_id=11f8731756fe4caca13efa61b71b11c5)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:43.871168 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08d974a-ba7f-47e0-a9c6-12002c26318e admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:43.894350 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1099/2224] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:46:43 2022] PUT /identity/v3/projects/34f32d48602e422b8d67ca1f4b2b0644/users/8a128dbe6fb245e78e5a2605ae6360e8/roles/11f8731756fe4caca13efa61b71b11c5 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 17:46:43.908255 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b35513-469d-41b3-9a02-ee671b6fe7e4 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:43.908542 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b35513-469d-41b3-9a02-ee671b6fe7e4 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:43.908821 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b35513-469d-41b3-9a02-ee671b6fe7e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:43.964334 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-94b35513-469d-41b3-9a02-ee671b6fe7e4 None None] MFA Rules not processed for user `8a128dbe6fb245e78e5a2605ae6360e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:46:43.965754 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-94b35513-469d-41b3-9a02-ee671b6fe7e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:46:43.988634 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:46:44.063151 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1126/2225] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:46:43 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:46:44.068181 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08d974a-ba7f-47e0-a9c6-12002c26318e admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:44.140433 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08d974a-ba7f-47e0-a9c6-12002c26318e admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:44.147868 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89aa1e0-2290-4293-a099-4faa2cf619ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:44.148725 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89aa1e0-2290-4293-a099-4faa2cf619ae admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:44.149007 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89aa1e0-2290-4293-a099-4faa2cf619ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:44.149320 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89aa1e0-2290-4293-a099-4faa2cf619ae admin admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:44.150025 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89aa1e0-2290-4293-a099-4faa2cf619ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:44.151906 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89aa1e0-2290-4293-a099-4faa2cf619ae admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:44.180873 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1100/2226] 10.222.0.89 () {60 vars in 1153 bytes} [Thu Jul 28 17:46:44 2022] POST /identity/v3/projects => generated 399 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 17:46:44.184889 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b35513-469d-41b3-9a02-ee671b6fe7e4 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:44.196012 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b35513-469d-41b3-9a02-ee671b6fe7e4 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:44.256491 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ff05f2-5005-4878-aa07-aac64ee468dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:44.256715 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ff05f2-5005-4878-aa07-aac64ee468dd admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:44.257139 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ff05f2-5005-4878-aa07-aac64ee468dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:44.257139 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ff05f2-5005-4878-aa07-aac64ee468dd admin admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:44.257833 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ff05f2-5005-4878-aa07-aac64ee468dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:44.259748 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ff05f2-5005-4878-aa07-aac64ee468dd admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:44.285809 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1127/2227] 10.222.0.89 () {60 vars in 1144 bytes} [Thu Jul 28 17:46:44 2022] POST /identity/v3/users => generated 306 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:46:44.287060 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89aa1e0-2290-4293-a099-4faa2cf619ae admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:44.301241 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89aa1e0-2290-4293-a099-4faa2cf619ae admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:44.308035 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892fba32-ab97-41a3-a796-4ff129acb610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:44.308669 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892fba32-ab97-41a3-a796-4ff129acb610 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:44.308951 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892fba32-ab97-41a3-a796-4ff129acb610 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:44.309260 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892fba32-ab97-41a3-a796-4ff129acb610 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:44.309963 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892fba32-ab97-41a3-a796-4ff129acb610 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:44.311654 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892fba32-ab97-41a3-a796-4ff129acb610 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:44.324409 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1101/2228] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:46:44 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:46:44.329475 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ff05f2-5005-4878-aa07-aac64ee468dd admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:44.342126 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ff05f2-5005-4878-aa07-aac64ee468dd admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:44.347373 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca637a2-406d-48cb-bd2d-f2f277528c9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:44.348034 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca637a2-406d-48cb-bd2d-f2f277528c9f admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:44.348316 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca637a2-406d-48cb-bd2d-f2f277528c9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:44.348588 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca637a2-406d-48cb-bd2d-f2f277528c9f admin admin] PATH_INFO: `/v3/projects/7ed2aee16d3b45b5a85d51fbdc82d30d/users/30bec220254f45a6addd150e1d730cda/roles/73e457231a824ea98c28d22610be640c` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:44.369478 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca637a2-406d-48cb-bd2d-f2f277528c9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ed2aee16d3b45b5a85d51fbdc82d30d, user_id=30bec220254f45a6addd150e1d730cda, role_id=73e457231a824ea98c28d22610be640c)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:44.372149 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca637a2-406d-48cb-bd2d-f2f277528c9f admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:44.393787 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1128/2229] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:46:44 2022] PUT /identity/v3/projects/7ed2aee16d3b45b5a85d51fbdc82d30d/users/30bec220254f45a6addd150e1d730cda/roles/73e457231a824ea98c28d22610be640c => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:46:44.402130 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892fba32-ab97-41a3-a796-4ff129acb610 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:44.413646 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892fba32-ab97-41a3-a796-4ff129acb610 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:44.486890 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b136b901-7d11-4820-9005-247d675f0114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:44.487652 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b136b901-7d11-4820-9005-247d675f0114 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:44.488041 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b136b901-7d11-4820-9005-247d675f0114 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:44.488420 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b136b901-7d11-4820-9005-247d675f0114 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:44.489302 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b136b901-7d11-4820-9005-247d675f0114 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:44.491057 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b136b901-7d11-4820-9005-247d675f0114 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:44.510074 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1102/2230] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:46:44 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:46:44.511349 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca637a2-406d-48cb-bd2d-f2f277528c9f admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:44.531783 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca637a2-406d-48cb-bd2d-f2f277528c9f admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:44.537749 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d058cc2f-6107-4f95-a247-6da510b2c1b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:44.538541 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d058cc2f-6107-4f95-a247-6da510b2c1b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:44.538921 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d058cc2f-6107-4f95-a247-6da510b2c1b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:44.539315 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d058cc2f-6107-4f95-a247-6da510b2c1b0 admin admin] PATH_INFO: `/v3/projects/7ed2aee16d3b45b5a85d51fbdc82d30d/users/30bec220254f45a6addd150e1d730cda/roles/73e457231a824ea98c28d22610be640c` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:44.541577 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d058cc2f-6107-4f95-a247-6da510b2c1b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ed2aee16d3b45b5a85d51fbdc82d30d, user_id=30bec220254f45a6addd150e1d730cda, role_id=73e457231a824ea98c28d22610be640c)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:44.544299 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d058cc2f-6107-4f95-a247-6da510b2c1b0 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:44.563185 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1129/2231] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:46:44 2022] PUT /identity/v3/projects/7ed2aee16d3b45b5a85d51fbdc82d30d/users/30bec220254f45a6addd150e1d730cda/roles/73e457231a824ea98c28d22610be640c => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:46:44.575074 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b79e57-5a04-41cc-879f-6b0dbf572c46 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:44.575356 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b79e57-5a04-41cc-879f-6b0dbf572c46 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:44.575644 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b79e57-5a04-41cc-879f-6b0dbf572c46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:44.598090 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:46:44.608929 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-36b79e57-5a04-41cc-879f-6b0dbf572c46 None None] MFA Rules not processed for user `30bec220254f45a6addd150e1d730cda`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:46:44.610912 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-36b79e57-5a04-41cc-879f-6b0dbf572c46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:46:44.705294 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1103/2232] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:46:44 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:46:44.716683 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d058cc2f-6107-4f95-a247-6da510b2c1b0 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:44.718677 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-d058cc2f-6107-4f95-a247-6da510b2c1b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:46:44.784591 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d058cc2f-6107-4f95-a247-6da510b2c1b0 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:44.788317 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6f6e87-b9d9-46b6-8a17-c37c791d393d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96d512d0c84240918dae6c4a3e8b7b79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:44.789088 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6f6e87-b9d9-46b6-8a17-c37c791d393d service cinder] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:44.789575 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6f6e87-b9d9-46b6-8a17-c37c791d393d service cinder] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:44.789768 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6f6e87-b9d9-46b6-8a17-c37c791d393d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:44.801238 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6f6e87-b9d9-46b6-8a17-c37c791d393d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:44.802686 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6f6e87-b9d9-46b6-8a17-c37c791d393d service cinder] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:44.825552 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1130/2233] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:46:44 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:46:44.925144 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b79e57-5a04-41cc-879f-6b0dbf572c46 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:44.935334 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b79e57-5a04-41cc-879f-6b0dbf572c46 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:44.939814 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f83f1f5-e44c-4e58-a4d3-a070b1768962 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96d512d0c84240918dae6c4a3e8b7b79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:44.940555 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f83f1f5-e44c-4e58-a4d3-a070b1768962 service cinder] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:44.940972 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f83f1f5-e44c-4e58-a4d3-a070b1768962 service cinder] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:44.941425 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f83f1f5-e44c-4e58-a4d3-a070b1768962 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:45.016485 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f83f1f5-e44c-4e58-a4d3-a070b1768962 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:45.018045 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f83f1f5-e44c-4e58-a4d3-a070b1768962 service cinder] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:45.027873 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1104/2234] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:46:44 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:46:46.042215 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf713ed5-6517-467c-bf35-e7040581ed5d None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:46.042703 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf713ed5-6517-467c-bf35-e7040581ed5d None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:46.042955 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf713ed5-6517-467c-bf35-e7040581ed5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:46.058860 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:46:46.065009 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-bf713ed5-6517-467c-bf35-e7040581ed5d None None] MFA Rules not processed for user `14327b8153ab4de782d4bcfef7681fc1`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:46:46.066555 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-bf713ed5-6517-467c-bf35-e7040581ed5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:46:46.128227 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1131/2235] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:46:46 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:46:50.235753 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f83f1f5-e44c-4e58-a4d3-a070b1768962 service cinder] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:50.308859 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f83f1f5-e44c-4e58-a4d3-a070b1768962 service cinder] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:50.313499 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152e9d27-c8a3-4164-b35b-d6b8017491fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:50.314147 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152e9d27-c8a3-4164-b35b-d6b8017491fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:50.314428 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152e9d27-c8a3-4164-b35b-d6b8017491fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:50.314703 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152e9d27-c8a3-4164-b35b-d6b8017491fb admin admin] PATH_INFO: `/v3/users/8a128dbe6fb245e78e5a2605ae6360e8` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:50.315884 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152e9d27-c8a3-4164-b35b-d6b8017491fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a128dbe6fb245e78e5a2605ae6360e8)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:50.317870 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152e9d27-c8a3-4164-b35b-d6b8017491fb admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:50.363484 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:46:50.375032 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-152e9d27-c8a3-4164-b35b-d6b8017491fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a128dbe6fb245e78e5a2605ae6360e8'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:50.383130 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-152e9d27-c8a3-4164-b35b-d6b8017491fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a128dbe6fb245e78e5a2605ae6360e8'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:50.402822 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-152e9d27-c8a3-4164-b35b-d6b8017491fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a128dbe6fb245e78e5a2605ae6360e8'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:50.405797 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-152e9d27-c8a3-4164-b35b-d6b8017491fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a128dbe6fb245e78e5a2605ae6360e8'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:50.406795 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-152e9d27-c8a3-4164-b35b-d6b8017491fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a128dbe6fb245e78e5a2605ae6360e8'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:50.432837 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1105/2236] 10.222.0.89 () {58 vars in 1224 bytes} [Thu Jul 28 17:46:50 2022] DELETE /identity/v3/users/8a128dbe6fb245e78e5a2605ae6360e8 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:46:50.437616 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa8e11d-78ac-403b-a3c8-fc91d3a0a5f4 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:50.438437 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa8e11d-78ac-403b-a3c8-fc91d3a0a5f4 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:50.438718 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa8e11d-78ac-403b-a3c8-fc91d3a0a5f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:50.463644 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-ffa8e11d-78ac-403b-a3c8-fc91d3a0a5f4 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:46:50.465957 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-ffa8e11d-78ac-403b-a3c8-fc91d3a0a5f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:46:50.542547 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1132/2237] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:46:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:46:50.553467 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152e9d27-c8a3-4164-b35b-d6b8017491fb admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:50.555011 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-152e9d27-c8a3-4164-b35b-d6b8017491fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:46:50.626334 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152e9d27-c8a3-4164-b35b-d6b8017491fb admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:50.630099 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7681ca-8b5e-4368-8942-5261c03e4d67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76817cd541294331b0023069092e8e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:50.630733 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7681ca-8b5e-4368-8942-5261c03e4d67 service neutron] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:50.631014 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7681ca-8b5e-4368-8942-5261c03e4d67 service neutron] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:50.631288 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7681ca-8b5e-4368-8942-5261c03e4d67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:50.642984 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7681ca-8b5e-4368-8942-5261c03e4d67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:50.644431 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7681ca-8b5e-4368-8942-5261c03e4d67 service neutron] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:50.655813 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1106/2238] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:46:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:46:50.937531 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa8e11d-78ac-403b-a3c8-fc91d3a0a5f4 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:50.939969 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-ffa8e11d-78ac-403b-a3c8-fc91d3a0a5f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:46:50.961470 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa8e11d-78ac-403b-a3c8-fc91d3a0a5f4 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:50.982782 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d97805-b94e-45ad-aadd-6201676d6f70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:50.984408 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d97805-b94e-45ad-aadd-6201676d6f70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:50.984804 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d97805-b94e-45ad-aadd-6201676d6f70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:50.985369 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d97805-b94e-45ad-aadd-6201676d6f70 admin admin] PATH_INFO: `/v3/users/63578fdf9c8c4b509f8d586b5f4e15b3` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:50.986756 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d97805-b94e-45ad-aadd-6201676d6f70 admin admin] RBAC: Authorizing `identity:delete_user(user_id=63578fdf9c8c4b509f8d586b5f4e15b3)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:50.988717 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7681ca-8b5e-4368-8942-5261c03e4d67 service neutron] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:50.990186 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-2f7681ca-8b5e-4368-8942-5261c03e4d67 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:46:50.990878 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d97805-b94e-45ad-aadd-6201676d6f70 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:51.025271 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7681ca-8b5e-4368-8942-5261c03e4d67 service neutron] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:51.041532 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14a0432-fb4e-4869-9642-0854c6252f99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:51.042125 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14a0432-fb4e-4869-9642-0854c6252f99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:51.042404 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14a0432-fb4e-4869-9642-0854c6252f99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:51.042679 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14a0432-fb4e-4869-9642-0854c6252f99 admin admin] PATH_INFO: `/v3/projects/34f32d48602e422b8d67ca1f4b2b0644` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:51.045206 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14a0432-fb4e-4869-9642-0854c6252f99 admin admin] RBAC: Authorizing `identity:delete_project(project_id=34f32d48602e422b8d67ca1f4b2b0644)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:51.055969 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14a0432-fb4e-4869-9642-0854c6252f99 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:51.116114 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-c7d97805-b94e-45ad-aadd-6201676d6f70 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63578fdf9c8c4b509f8d586b5f4e15b3'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:51.141237 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-c7d97805-b94e-45ad-aadd-6201676d6f70 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63578fdf9c8c4b509f8d586b5f4e15b3'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:51.142111 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-c7d97805-b94e-45ad-aadd-6201676d6f70 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63578fdf9c8c4b509f8d586b5f4e15b3'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:51.166394 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-c7d97805-b94e-45ad-aadd-6201676d6f70 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63578fdf9c8c4b509f8d586b5f4e15b3'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:51.184508 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-c7d97805-b94e-45ad-aadd-6201676d6f70 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63578fdf9c8c4b509f8d586b5f4e15b3'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:51.193831 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1133/2239] 10.222.0.89 () {58 vars in 1224 bytes} [Thu Jul 28 17:46:50 2022] DELETE /identity/v3/users/63578fdf9c8c4b509f8d586b5f4e15b3 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:46:51.197456 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-c14a0432-fb4e-4869-9642-0854c6252f99 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34f32d48602e422b8d67ca1f4b2b0644', 'request_id': 'req-c14a0432-fb4e-4869-9642-0854c6252f99'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:51.200020 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a53195-87fa-4f9a-ba68-2ce9887901ca None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:51.200231 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a53195-87fa-4f9a-ba68-2ce9887901ca None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:51.200685 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a53195-87fa-4f9a-ba68-2ce9887901ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:51.205183 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-c14a0432-fb4e-4869-9642-0854c6252f99 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34f32d48602e422b8d67ca1f4b2b0644', 'request_id': 'req-c14a0432-fb4e-4869-9642-0854c6252f99'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:51.212070 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-c14a0432-fb4e-4869-9642-0854c6252f99 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34f32d48602e422b8d67ca1f4b2b0644', 'request_id': 'req-c14a0432-fb4e-4869-9642-0854c6252f99'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:51.228141 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-c14a0432-fb4e-4869-9642-0854c6252f99 admin admin] The token cache is being invalidate because project 34f32d48602e422b8d67ca1f4b2b0644 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80325) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 17:46:51.228482 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-c14a0432-fb4e-4869-9642-0854c6252f99 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:51.229198 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-c14a0432-fb4e-4869-9642-0854c6252f99 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:51.229893 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-a2a53195-87fa-4f9a-ba68-2ce9887901ca None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:46:51.231139 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1107/2240] 10.222.0.89 () {58 vars in 1233 bytes} [Thu Jul 28 17:46:50 2022] DELETE /identity/v3/projects/34f32d48602e422b8d67ca1f4b2b0644 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 17:46:51.231453 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-a2a53195-87fa-4f9a-ba68-2ce9887901ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:46:51.235879 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14a0432-fb4e-4869-9642-0854c6252f99 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:51.237040 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-c14a0432-fb4e-4869-9642-0854c6252f99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:46:51.258581 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:46:51.291820 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:46:51.317085 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1134/2241] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:46:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 17:46:51.333889 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a53195-87fa-4f9a-ba68-2ce9887901ca None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:51.334823 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14a0432-fb4e-4869-9642-0854c6252f99 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:51.336054 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-a2a53195-87fa-4f9a-ba68-2ce9887901ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:46:51.349543 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368b9e36-9425-4c1d-a15f-8b11fce8c166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:51.349543 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368b9e36-9425-4c1d-a15f-8b11fce8c166 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:51.349543 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368b9e36-9425-4c1d-a15f-8b11fce8c166 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:51.349543 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368b9e36-9425-4c1d-a15f-8b11fce8c166 admin admin] PATH_INFO: `/v3/users/30bec220254f45a6addd150e1d730cda` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:51.349831 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368b9e36-9425-4c1d-a15f-8b11fce8c166 admin admin] RBAC: Authorizing `identity:delete_user(user_id=30bec220254f45a6addd150e1d730cda)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:51.349831 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368b9e36-9425-4c1d-a15f-8b11fce8c166 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:51.422108 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a53195-87fa-4f9a-ba68-2ce9887901ca None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:51.426289 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62b0d63-14de-4422-8c46-09c399608759 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76817cd541294331b0023069092e8e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:51.426741 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62b0d63-14de-4422-8c46-09c399608759 service neutron] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:51.427008 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62b0d63-14de-4422-8c46-09c399608759 service neutron] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:51.427271 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62b0d63-14de-4422-8c46-09c399608759 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:51.440921 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62b0d63-14de-4422-8c46-09c399608759 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:51.443183 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62b0d63-14de-4422-8c46-09c399608759 service neutron] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:51.455617 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1135/2242] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:46:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:46:51.471847 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-368b9e36-9425-4c1d-a15f-8b11fce8c166 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30bec220254f45a6addd150e1d730cda'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:51.492726 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-368b9e36-9425-4c1d-a15f-8b11fce8c166 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30bec220254f45a6addd150e1d730cda'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:51.528577 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-368b9e36-9425-4c1d-a15f-8b11fce8c166 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30bec220254f45a6addd150e1d730cda'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:51.529451 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-368b9e36-9425-4c1d-a15f-8b11fce8c166 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30bec220254f45a6addd150e1d730cda'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:51.530339 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-368b9e36-9425-4c1d-a15f-8b11fce8c166 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30bec220254f45a6addd150e1d730cda'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:51.573566 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1108/2243] 10.222.0.89 () {58 vars in 1224 bytes} [Thu Jul 28 17:46:51 2022] DELETE /identity/v3/users/30bec220254f45a6addd150e1d730cda => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:46:52.028624 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62b0d63-14de-4422-8c46-09c399608759 service neutron] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:52.031833 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-e62b0d63-14de-4422-8c46-09c399608759 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:46:52.046754 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62b0d63-14de-4422-8c46-09c399608759 service neutron] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:52.114621 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368b9e36-9425-4c1d-a15f-8b11fce8c166 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:52.114621 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-368b9e36-9425-4c1d-a15f-8b11fce8c166 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:46:52.161942 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80defb5-92c4-4023-bf12-db4c83d032bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:52.162854 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80defb5-92c4-4023-bf12-db4c83d032bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:52.163198 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80defb5-92c4-4023-bf12-db4c83d032bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:52.163484 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80defb5-92c4-4023-bf12-db4c83d032bf admin admin] PATH_INFO: `/v3/projects/47a7b20146624d4aa25006a7b1c05b97` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:52.172699 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80defb5-92c4-4023-bf12-db4c83d032bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=47a7b20146624d4aa25006a7b1c05b97)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:52.179572 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80defb5-92c4-4023-bf12-db4c83d032bf admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:52.228239 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368b9e36-9425-4c1d-a15f-8b11fce8c166 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:52.245605 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-f80defb5-92c4-4023-bf12-db4c83d032bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47a7b20146624d4aa25006a7b1c05b97', 'request_id': 'req-f80defb5-92c4-4023-bf12-db4c83d032bf'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:52.262208 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-f80defb5-92c4-4023-bf12-db4c83d032bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47a7b20146624d4aa25006a7b1c05b97', 'request_id': 'req-f80defb5-92c4-4023-bf12-db4c83d032bf'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:52.273548 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-f80defb5-92c4-4023-bf12-db4c83d032bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47a7b20146624d4aa25006a7b1c05b97', 'request_id': 'req-f80defb5-92c4-4023-bf12-db4c83d032bf'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:52.288046 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:46:52.293453 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-f80defb5-92c4-4023-bf12-db4c83d032bf admin admin] The token cache is being invalidate because project 47a7b20146624d4aa25006a7b1c05b97 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80324) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 17:46:52.294045 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-f80defb5-92c4-4023-bf12-db4c83d032bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:52.295210 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-f80defb5-92c4-4023-bf12-db4c83d032bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:52.297802 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1136/2244] 10.222.0.89 () {58 vars in 1233 bytes} [Thu Jul 28 17:46:52 2022] DELETE /identity/v3/projects/47a7b20146624d4aa25006a7b1c05b97 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:46:52.303025 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c51c5e-e18e-4963-bfb1-7d731b1d05db None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:52.303438 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c51c5e-e18e-4963-bfb1-7d731b1d05db None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:52.303716 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c51c5e-e18e-4963-bfb1-7d731b1d05db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:52.313844 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74038242-b154-406a-bc1b-3cec86c1a5df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:52.314506 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74038242-b154-406a-bc1b-3cec86c1a5df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:52.314790 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74038242-b154-406a-bc1b-3cec86c1a5df admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:52.315382 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74038242-b154-406a-bc1b-3cec86c1a5df admin admin] PATH_INFO: `/v3/projects/7ed2aee16d3b45b5a85d51fbdc82d30d` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:52.317068 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74038242-b154-406a-bc1b-3cec86c1a5df admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ed2aee16d3b45b5a85d51fbdc82d30d)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:52.319439 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74038242-b154-406a-bc1b-3cec86c1a5df admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:52.328473 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:46:52.329970 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-e6c51c5e-e18e-4963-bfb1-7d731b1d05db None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:46:52.331057 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-e6c51c5e-e18e-4963-bfb1-7d731b1d05db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:46:52.340268 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1137/2245] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:46:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 17:46:52.349486 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effa3ef9-5d6e-4e94-b132-2553784d5387 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:52.349779 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effa3ef9-5d6e-4e94-b132-2553784d5387 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:52.350052 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effa3ef9-5d6e-4e94-b132-2553784d5387 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:52.377386 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-effa3ef9-5d6e-4e94-b132-2553784d5387 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:46:52.379016 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-effa3ef9-5d6e-4e94-b132-2553784d5387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:46:52.379957 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-74038242-b154-406a-bc1b-3cec86c1a5df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ed2aee16d3b45b5a85d51fbdc82d30d', 'request_id': 'req-74038242-b154-406a-bc1b-3cec86c1a5df'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:52.386977 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-74038242-b154-406a-bc1b-3cec86c1a5df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ed2aee16d3b45b5a85d51fbdc82d30d', 'request_id': 'req-74038242-b154-406a-bc1b-3cec86c1a5df'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:52.394121 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-74038242-b154-406a-bc1b-3cec86c1a5df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ed2aee16d3b45b5a85d51fbdc82d30d', 'request_id': 'req-74038242-b154-406a-bc1b-3cec86c1a5df'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:52.412156 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-74038242-b154-406a-bc1b-3cec86c1a5df admin admin] The token cache is being invalidate because project 7ed2aee16d3b45b5a85d51fbdc82d30d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80325) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 17:46:52.412456 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-74038242-b154-406a-bc1b-3cec86c1a5df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:52.413131 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-74038242-b154-406a-bc1b-3cec86c1a5df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:52.417199 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1109/2246] 10.222.0.89 () {58 vars in 1233 bytes} [Thu Jul 28 17:46:52 2022] DELETE /identity/v3/projects/7ed2aee16d3b45b5a85d51fbdc82d30d => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:46:52.419309 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410cf3f2-1588-4054-b719-adea32cc6fae None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:52.420052 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410cf3f2-1588-4054-b719-adea32cc6fae None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:52.420653 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410cf3f2-1588-4054-b719-adea32cc6fae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:52.448479 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-410cf3f2-1588-4054-b719-adea32cc6fae None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:46:52.449759 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-410cf3f2-1588-4054-b719-adea32cc6fae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:46:52.455023 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1138/2247] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:46:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 17:46:52.458943 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1110/2248] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:46:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 17:46:52.461620 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effa3ef9-5d6e-4e94-b132-2553784d5387 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:52.466371 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fbb071-785e-44c9-8b56-0e2b1774a908 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:52.466660 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fbb071-785e-44c9-8b56-0e2b1774a908 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:52.466936 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fbb071-785e-44c9-8b56-0e2b1774a908 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:52.476079 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effa3ef9-5d6e-4e94-b132-2553784d5387 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:52.482159 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8d9163-b2f1-4ddf-b145-24279597e348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:52.482788 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8d9163-b2f1-4ddf-b145-24279597e348 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:52.483060 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8d9163-b2f1-4ddf-b145-24279597e348 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:52.484367 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8d9163-b2f1-4ddf-b145-24279597e348 admin admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:52.485229 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8d9163-b2f1-4ddf-b145-24279597e348 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:46:52.485516 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8d9163-b2f1-4ddf-b145-24279597e348 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:52.487071 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-01fbb071-785e-44c9-8b56-0e2b1774a908 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:46:52.487286 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8d9163-b2f1-4ddf-b145-24279597e348 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:52.488292 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-01fbb071-785e-44c9-8b56-0e2b1774a908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:46:52.496463 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1111/2249] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:46:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 17:46:52.500742 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1139/2250] 10.222.0.89 () {58 vars in 1153 bytes} [Thu Jul 28 17:46:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:46:52.501748 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fbb071-785e-44c9-8b56-0e2b1774a908 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:52.506138 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8d9163-b2f1-4ddf-b145-24279597e348 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:52.511475 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fbb071-785e-44c9-8b56-0e2b1774a908 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:52.516813 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db7dde6-e1a5-4ce6-9461-9acecab21c46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:52.517229 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8d9163-b2f1-4ddf-b145-24279597e348 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:52.517389 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db7dde6-e1a5-4ce6-9461-9acecab21c46 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:52.517671 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db7dde6-e1a5-4ce6-9461-9acecab21c46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:52.518532 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db7dde6-e1a5-4ce6-9461-9acecab21c46 admin admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:52.518894 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db7dde6-e1a5-4ce6-9461-9acecab21c46 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:46:52.519208 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db7dde6-e1a5-4ce6-9461-9acecab21c46 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:52.520806 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db7dde6-e1a5-4ce6-9461-9acecab21c46 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:52.522376 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea56eb14-1aea-4e57-ad76-ccb9209e26a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:52.522992 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea56eb14-1aea-4e57-ad76-ccb9209e26a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:52.523275 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea56eb14-1aea-4e57-ad76-ccb9209e26a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:52.523958 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea56eb14-1aea-4e57-ad76-ccb9209e26a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:52.524257 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea56eb14-1aea-4e57-ad76-ccb9209e26a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:52.526189 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea56eb14-1aea-4e57-ad76-ccb9209e26a5 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:52.538833 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1112/2251] 10.222.0.89 () {58 vars in 1153 bytes} [Thu Jul 28 17:46:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:46:52.539488 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db7dde6-e1a5-4ce6-9461-9acecab21c46 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:52.550547 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db7dde6-e1a5-4ce6-9461-9acecab21c46 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:52.566961 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1140/2252] 10.222.0.89 () {60 vars in 1153 bytes} [Thu Jul 28 17:46:52 2022] POST /identity/v3/projects => generated 385 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 17:46:52.570958 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea56eb14-1aea-4e57-ad76-ccb9209e26a5 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:52.572226 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f11976-0455-4203-8299-1ad06d200d2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:52.572949 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f11976-0455-4203-8299-1ad06d200d2f admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:52.573334 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f11976-0455-4203-8299-1ad06d200d2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:52.573619 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f11976-0455-4203-8299-1ad06d200d2f admin admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:52.574288 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f11976-0455-4203-8299-1ad06d200d2f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:52.576221 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f11976-0455-4203-8299-1ad06d200d2f admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:52.616971 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1113/2253] 10.222.0.89 () {60 vars in 1153 bytes} [Thu Jul 28 17:46:52 2022] POST /identity/v3/projects => generated 383 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:46:52.621021 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f11976-0455-4203-8299-1ad06d200d2f admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:52.647807 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea56eb14-1aea-4e57-ad76-ccb9209e26a5 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:52.699722 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f11976-0455-4203-8299-1ad06d200d2f admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:52.707502 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af482e0d-ae10-472b-bcdc-074c979afdc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:52.708330 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af482e0d-ae10-472b-bcdc-074c979afdc9 admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:52.708700 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af482e0d-ae10-472b-bcdc-074c979afdc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:52.709078 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af482e0d-ae10-472b-bcdc-074c979afdc9 admin admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:52.710012 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af482e0d-ae10-472b-bcdc-074c979afdc9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:52.712160 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af482e0d-ae10-472b-bcdc-074c979afdc9 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:52.732359 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652fce13-46b3-4fef-95bd-376bbba62280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:52.732976 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652fce13-46b3-4fef-95bd-376bbba62280 admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:52.733625 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652fce13-46b3-4fef-95bd-376bbba62280 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:52.733928 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652fce13-46b3-4fef-95bd-376bbba62280 admin admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:52.734712 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652fce13-46b3-4fef-95bd-376bbba62280 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:52.736895 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652fce13-46b3-4fef-95bd-376bbba62280 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:52.740077 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1114/2254] 10.222.0.89 () {60 vars in 1144 bytes} [Thu Jul 28 17:46:52 2022] POST /identity/v3/users => generated 292 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 17:46:52.743920 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af482e0d-ae10-472b-bcdc-074c979afdc9 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:52.749873 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:46:52.755508 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af482e0d-ae10-472b-bcdc-074c979afdc9 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:52.765400 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc69667-9c88-4901-bf59-dcecfe6cbcba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:52.769758 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1141/2255] 10.222.0.89 () {60 vars in 1144 bytes} [Thu Jul 28 17:46:52 2022] POST /identity/v3/users => generated 293 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 17:46:52.769897 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc69667-9c88-4901-bf59-dcecfe6cbcba admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:52.769897 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc69667-9c88-4901-bf59-dcecfe6cbcba admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:52.769897 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc69667-9c88-4901-bf59-dcecfe6cbcba admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:52.771078 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652fce13-46b3-4fef-95bd-376bbba62280 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:52.772144 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc69667-9c88-4901-bf59-dcecfe6cbcba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:52.774465 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc69667-9c88-4901-bf59-dcecfe6cbcba admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:52.793041 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1115/2256] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:46:52 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 17:46:52.793980 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc69667-9c88-4901-bf59-dcecfe6cbcba admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:52.799065 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652fce13-46b3-4fef-95bd-376bbba62280 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:52.806826 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc69667-9c88-4901-bf59-dcecfe6cbcba admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:52.812416 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfeeae0-e5e5-41df-87d8-584fc447f809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:52.813793 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfeeae0-e5e5-41df-87d8-584fc447f809 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:52.814210 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfeeae0-e5e5-41df-87d8-584fc447f809 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:52.814655 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfeeae0-e5e5-41df-87d8-584fc447f809 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:52.815647 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfeeae0-e5e5-41df-87d8-584fc447f809 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:52.817986 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfeeae0-e5e5-41df-87d8-584fc447f809 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:52.818573 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ec0c99-4a3c-4739-88c1-3910235724fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:52.819715 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ec0c99-4a3c-4739-88c1-3910235724fe admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:52.819998 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ec0c99-4a3c-4739-88c1-3910235724fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:52.820269 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ec0c99-4a3c-4739-88c1-3910235724fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:52.821199 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ec0c99-4a3c-4739-88c1-3910235724fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:52.833826 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ec0c99-4a3c-4739-88c1-3910235724fe admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:52.848301 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1116/2257] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:46:52 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:46:52.848301 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfeeae0-e5e5-41df-87d8-584fc447f809 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:52.853600 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1142/2258] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:46:52 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:46:52.859918 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ec0c99-4a3c-4739-88c1-3910235724fe admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:52.868636 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfeeae0-e5e5-41df-87d8-584fc447f809 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:52.878394 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ec0c99-4a3c-4739-88c1-3910235724fe admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:52.882221 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45e3929-af00-4572-a19e-e5c542a85a96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:52.884002 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2452758-bcda-40a0-9f36-b49def69f264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:52.884129 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45e3929-af00-4572-a19e-e5c542a85a96 admin admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:52.885198 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45e3929-af00-4572-a19e-e5c542a85a96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:52.885582 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45e3929-af00-4572-a19e-e5c542a85a96 admin admin] PATH_INFO: `/v3/projects/d9cbb5973aed44eab8e99ed8aceb1241/users/905c91e9aabb4d1e9f2c0a68f9c1870d/roles/11f8731756fe4caca13efa61b71b11c5` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:52.885986 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2452758-bcda-40a0-9f36-b49def69f264 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:52.886564 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2452758-bcda-40a0-9f36-b49def69f264 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:52.887015 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2452758-bcda-40a0-9f36-b49def69f264 admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:52.888141 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2452758-bcda-40a0-9f36-b49def69f264 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:52.890794 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2452758-bcda-40a0-9f36-b49def69f264 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:52.910278 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:46:52.917882 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45e3929-af00-4572-a19e-e5c542a85a96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9cbb5973aed44eab8e99ed8aceb1241, user_id=905c91e9aabb4d1e9f2c0a68f9c1870d, role_id=11f8731756fe4caca13efa61b71b11c5)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:52.921151 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45e3929-af00-4572-a19e-e5c542a85a96 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:52.928454 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1143/2259] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:46:52 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:46:52.932779 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2452758-bcda-40a0-9f36-b49def69f264 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:52.946168 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2452758-bcda-40a0-9f36-b49def69f264 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:52.953245 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1117/2260] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:46:52 2022] PUT /identity/v3/projects/d9cbb5973aed44eab8e99ed8aceb1241/users/905c91e9aabb4d1e9f2c0a68f9c1870d/roles/11f8731756fe4caca13efa61b71b11c5 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:46:52.960824 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d626276d-78eb-4448-8af6-04ab6bee4308 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:52.961232 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d626276d-78eb-4448-8af6-04ab6bee4308 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:52.961843 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d626276d-78eb-4448-8af6-04ab6bee4308 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:52.998300 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-d626276d-78eb-4448-8af6-04ab6bee4308 None None] MFA Rules not processed for user `905c91e9aabb4d1e9f2c0a68f9c1870d`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:46:53.000709 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-d626276d-78eb-4448-8af6-04ab6bee4308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:46:53.021738 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b499c7b8-b9af-4e42-9585-809709e1dc88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:53.022333 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b499c7b8-b9af-4e42-9585-809709e1dc88 admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:53.022619 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b499c7b8-b9af-4e42-9585-809709e1dc88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:53.022898 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b499c7b8-b9af-4e42-9585-809709e1dc88 admin admin] PATH_INFO: `/v3/projects/dd646d06c265400aa0e578563e4e7cb3/users/9ddb8f47fdd343e8bafb283a3370c2e9/roles/11f8731756fe4caca13efa61b71b11c5` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:53.039298 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b499c7b8-b9af-4e42-9585-809709e1dc88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd646d06c265400aa0e578563e4e7cb3, user_id=9ddb8f47fdd343e8bafb283a3370c2e9, role_id=11f8731756fe4caca13efa61b71b11c5)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:53.042086 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b499c7b8-b9af-4e42-9585-809709e1dc88 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:53.056285 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1144/2261] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:46:52 2022] PUT /identity/v3/projects/dd646d06c265400aa0e578563e4e7cb3/users/9ddb8f47fdd343e8bafb283a3370c2e9/roles/11f8731756fe4caca13efa61b71b11c5 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 17:46:53.068160 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8d8d64-2622-4b2a-aed5-e1d43b78c10e None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:53.068445 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8d8d64-2622-4b2a-aed5-e1d43b78c10e None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:53.068755 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8d8d64-2622-4b2a-aed5-e1d43b78c10e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:53.076370 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1118/2262] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:46:52 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:46:53.081251 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d626276d-78eb-4448-8af6-04ab6bee4308 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:53.091268 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d626276d-78eb-4448-8af6-04ab6bee4308 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:53.097485 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-7f8d8d64-2622-4b2a-aed5-e1d43b78c10e None None] MFA Rules not processed for user `9ddb8f47fdd343e8bafb283a3370c2e9`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:46:53.098746 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-7f8d8d64-2622-4b2a-aed5-e1d43b78c10e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:46:53.150961 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57f6337-ac59-40f3-bef3-adda40fba087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:53.151722 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57f6337-ac59-40f3-bef3-adda40fba087 admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:53.152008 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57f6337-ac59-40f3-bef3-adda40fba087 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:53.152279 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57f6337-ac59-40f3-bef3-adda40fba087 admin admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:53.152952 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57f6337-ac59-40f3-bef3-adda40fba087 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:53.154890 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57f6337-ac59-40f3-bef3-adda40fba087 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:53.167208 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1145/2263] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:46:53 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 17:46:53.172129 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8d8d64-2622-4b2a-aed5-e1d43b78c10e None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:53.182516 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1119/2264] 10.222.0.89 () {60 vars in 1153 bytes} [Thu Jul 28 17:46:53 2022] POST /identity/v3/projects => generated 383 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 17:46:53.183595 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8d8d64-2622-4b2a-aed5-e1d43b78c10e None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:53.187271 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57f6337-ac59-40f3-bef3-adda40fba087 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:53.226003 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:46:53.243707 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40ff2fb-a5cc-4f74-985b-050285c2322e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:53.244327 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40ff2fb-a5cc-4f74-985b-050285c2322e admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:53.244615 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40ff2fb-a5cc-4f74-985b-050285c2322e admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:53.244887 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40ff2fb-a5cc-4f74-985b-050285c2322e admin admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:53.245605 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40ff2fb-a5cc-4f74-985b-050285c2322e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:53.247538 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40ff2fb-a5cc-4f74-985b-050285c2322e admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:53.250057 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57f6337-ac59-40f3-bef3-adda40fba087 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:53.254815 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a47e71e-219c-402c-851a-82129f3f7de5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:53.255440 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a47e71e-219c-402c-851a-82129f3f7de5 admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:53.255728 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a47e71e-219c-402c-851a-82129f3f7de5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:53.256004 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a47e71e-219c-402c-851a-82129f3f7de5 admin admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:53.256672 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a47e71e-219c-402c-851a-82129f3f7de5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:53.258646 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a47e71e-219c-402c-851a-82129f3f7de5 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:53.270386 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:46:53.279758 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1146/2265] 10.222.0.89 () {60 vars in 1153 bytes} [Thu Jul 28 17:46:53 2022] POST /identity/v3/projects => generated 385 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:46:53.281086 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40ff2fb-a5cc-4f74-985b-050285c2322e admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:53.284595 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1120/2266] 10.222.0.89 () {60 vars in 1144 bytes} [Thu Jul 28 17:46:53 2022] POST /identity/v3/users => generated 298 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 17:46:53.288576 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a47e71e-219c-402c-851a-82129f3f7de5 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:53.299684 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a47e71e-219c-402c-851a-82129f3f7de5 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:53.361521 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40ff2fb-a5cc-4f74-985b-050285c2322e admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:53.367010 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef85f4c5-e5a1-4fb7-a86b-5033b09ee7ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:53.368046 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef85f4c5-e5a1-4fb7-a86b-5033b09ee7ca admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:53.368046 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef85f4c5-e5a1-4fb7-a86b-5033b09ee7ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:53.368302 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a857a1-373f-4bac-a2b7-25780021b003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:53.368742 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef85f4c5-e5a1-4fb7-a86b-5033b09ee7ca admin admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:53.368838 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a857a1-373f-4bac-a2b7-25780021b003 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:53.369109 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a857a1-373f-4bac-a2b7-25780021b003 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:53.369454 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a857a1-373f-4bac-a2b7-25780021b003 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:53.369585 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef85f4c5-e5a1-4fb7-a86b-5033b09ee7ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:53.370159 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a857a1-373f-4bac-a2b7-25780021b003 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:53.371803 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef85f4c5-e5a1-4fb7-a86b-5033b09ee7ca admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:53.371925 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a857a1-373f-4bac-a2b7-25780021b003 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:53.390266 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1121/2267] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:46:53 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:46:53.392624 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a857a1-373f-4bac-a2b7-25780021b003 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:53.400571 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1147/2268] 10.222.0.89 () {60 vars in 1144 bytes} [Thu Jul 28 17:46:53 2022] POST /identity/v3/users => generated 293 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 17:46:53.401734 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef85f4c5-e5a1-4fb7-a86b-5033b09ee7ca admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:53.402936 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a857a1-373f-4bac-a2b7-25780021b003 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:53.409779 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5af2097-21da-47fd-885c-3be8ef261eeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:53.410463 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5af2097-21da-47fd-885c-3be8ef261eeb admin admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:53.410731 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5af2097-21da-47fd-885c-3be8ef261eeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:53.411456 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5af2097-21da-47fd-885c-3be8ef261eeb admin admin] PATH_INFO: `/v3/projects/d9dd7a97cfe540a38dccbad57810540d/users/937d966df5be4fb38bae46d127396edb/roles/73e457231a824ea98c28d22610be640c` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:53.414073 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef85f4c5-e5a1-4fb7-a86b-5033b09ee7ca admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:53.421599 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d717ed2c-496a-40f6-a829-7f5055c865fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:53.422396 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d717ed2c-496a-40f6-a829-7f5055c865fe admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:53.423178 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d717ed2c-496a-40f6-a829-7f5055c865fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:53.423460 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d717ed2c-496a-40f6-a829-7f5055c865fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:53.424326 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d717ed2c-496a-40f6-a829-7f5055c865fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:53.425980 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d717ed2c-496a-40f6-a829-7f5055c865fe admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:53.429790 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5af2097-21da-47fd-885c-3be8ef261eeb admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9dd7a97cfe540a38dccbad57810540d, user_id=937d966df5be4fb38bae46d127396edb, role_id=73e457231a824ea98c28d22610be640c)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:53.432487 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5af2097-21da-47fd-885c-3be8ef261eeb admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:53.438516 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1148/2269] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:46:53 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:46:53.443119 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d717ed2c-496a-40f6-a829-7f5055c865fe admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:53.453334 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1122/2270] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:46:53 2022] PUT /identity/v3/projects/d9dd7a97cfe540a38dccbad57810540d/users/937d966df5be4fb38bae46d127396edb/roles/73e457231a824ea98c28d22610be640c => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:46:53.456476 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d717ed2c-496a-40f6-a829-7f5055c865fe admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:53.458257 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5af2097-21da-47fd-885c-3be8ef261eeb admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:53.536101 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8410cd27-0b86-4203-b63e-a4a2c984f942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:53.537062 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8410cd27-0b86-4203-b63e-a4a2c984f942 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:53.537574 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8410cd27-0b86-4203-b63e-a4a2c984f942 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:53.537999 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8410cd27-0b86-4203-b63e-a4a2c984f942 admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:53.539157 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8410cd27-0b86-4203-b63e-a4a2c984f942 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:53.540887 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5af2097-21da-47fd-885c-3be8ef261eeb admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:53.541500 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8410cd27-0b86-4203-b63e-a4a2c984f942 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:53.546110 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed187c9f-66ac-483d-b176-8754ceabba52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:53.547335 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed187c9f-66ac-483d-b176-8754ceabba52 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:53.547908 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed187c9f-66ac-483d-b176-8754ceabba52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:53.548213 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed187c9f-66ac-483d-b176-8754ceabba52 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:53.548949 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed187c9f-66ac-483d-b176-8754ceabba52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:53.550816 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed187c9f-66ac-483d-b176-8754ceabba52 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:53.559495 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1149/2271] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:46:53 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 17:46:53.560987 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8410cd27-0b86-4203-b63e-a4a2c984f942 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:53.562198 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1123/2272] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:46:53 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:46:53.566382 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed187c9f-66ac-483d-b176-8754ceabba52 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:53.573379 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8410cd27-0b86-4203-b63e-a4a2c984f942 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:53.581433 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6c46af-aa6b-495c-a12b-6a4c36c0a2af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:53.582122 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6c46af-aa6b-495c-a12b-6a4c36c0a2af admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:53.582401 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6c46af-aa6b-495c-a12b-6a4c36c0a2af admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:53.582678 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6c46af-aa6b-495c-a12b-6a4c36c0a2af admin admin] PATH_INFO: `/v3/projects/6f70e34bfec746159e123060fac97115/users/39649ddac168401c98df597bf2a934d9/roles/11f8731756fe4caca13efa61b71b11c5` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:53.607259 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6c46af-aa6b-495c-a12b-6a4c36c0a2af admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f70e34bfec746159e123060fac97115, user_id=39649ddac168401c98df597bf2a934d9, role_id=11f8731756fe4caca13efa61b71b11c5)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:53.611225 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6c46af-aa6b-495c-a12b-6a4c36c0a2af admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:53.621436 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed187c9f-66ac-483d-b176-8754ceabba52 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:53.629900 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1150/2273] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:46:53 2022] PUT /identity/v3/projects/6f70e34bfec746159e123060fac97115/users/39649ddac168401c98df597bf2a934d9/roles/11f8731756fe4caca13efa61b71b11c5 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:46:53.640788 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb25b6fb-d648-462c-9730-e4ef9c92e32e None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:53.641097 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb25b6fb-d648-462c-9730-e4ef9c92e32e None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:53.641432 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb25b6fb-d648-462c-9730-e4ef9c92e32e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:53.691183 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-eb25b6fb-d648-462c-9730-e4ef9c92e32e None None] MFA Rules not processed for user `39649ddac168401c98df597bf2a934d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:46:53.693186 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-eb25b6fb-d648-462c-9730-e4ef9c92e32e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:46:53.719199 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:46:53.758989 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3a9371-0962-494a-a5bc-557e879da7f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:53.759675 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3a9371-0962-494a-a5bc-557e879da7f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:53.760131 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3a9371-0962-494a-a5bc-557e879da7f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:53.760405 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3a9371-0962-494a-a5bc-557e879da7f5 admin admin] PATH_INFO: `/v3/projects/d9dd7a97cfe540a38dccbad57810540d/users/937d966df5be4fb38bae46d127396edb/roles/73e457231a824ea98c28d22610be640c` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:53.763740 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3a9371-0962-494a-a5bc-557e879da7f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9dd7a97cfe540a38dccbad57810540d, user_id=937d966df5be4fb38bae46d127396edb, role_id=73e457231a824ea98c28d22610be640c)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:53.773576 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3a9371-0962-494a-a5bc-557e879da7f5 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:53.778082 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:46:53.790815 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1124/2274] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:46:53 2022] PUT /identity/v3/projects/d9dd7a97cfe540a38dccbad57810540d/users/937d966df5be4fb38bae46d127396edb/roles/73e457231a824ea98c28d22610be640c => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 17:46:53.804239 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068d78e0-8995-4705-a4fc-57e644ccf2e7 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:53.804537 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068d78e0-8995-4705-a4fc-57e644ccf2e7 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:53.804819 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068d78e0-8995-4705-a4fc-57e644ccf2e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:53.831238 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1151/2275] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:46:53 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 17:46:53.837131 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb25b6fb-d648-462c-9730-e4ef9c92e32e None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:53.850933 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-068d78e0-8995-4705-a4fc-57e644ccf2e7 None None] MFA Rules not processed for user `937d966df5be4fb38bae46d127396edb`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:46:53.852169 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-068d78e0-8995-4705-a4fc-57e644ccf2e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:46:53.852861 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb25b6fb-d648-462c-9730-e4ef9c92e32e None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:53.933597 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb263976-270d-4fa8-b291-0f2949c718fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:53.934628 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb263976-270d-4fa8-b291-0f2949c718fa admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:53.935090 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb263976-270d-4fa8-b291-0f2949c718fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:53.935542 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb263976-270d-4fa8-b291-0f2949c718fa admin admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:53.935753 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1125/2276] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:46:53 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 17:46:53.936917 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb263976-270d-4fa8-b291-0f2949c718fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:53.939339 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb263976-270d-4fa8-b291-0f2949c718fa admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:53.957489 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068d78e0-8995-4705-a4fc-57e644ccf2e7 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:53.962342 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-068d78e0-8995-4705-a4fc-57e644ccf2e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:46:54.006409 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1152/2277] 10.222.0.89 () {60 vars in 1153 bytes} [Thu Jul 28 17:46:53 2022] POST /identity/v3/projects => generated 385 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 17:46:54.009293 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb263976-270d-4fa8-b291-0f2949c718fa admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:54.096188 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068d78e0-8995-4705-a4fc-57e644ccf2e7 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:54.129715 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb263976-270d-4fa8-b291-0f2949c718fa admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:54.135942 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65dadf7-039f-477d-abc7-d9e3cb98bae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:54.136737 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65dadf7-039f-477d-abc7-d9e3cb98bae0 admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:54.137186 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65dadf7-039f-477d-abc7-d9e3cb98bae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:54.137872 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65dadf7-039f-477d-abc7-d9e3cb98bae0 admin admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:54.138675 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65dadf7-039f-477d-abc7-d9e3cb98bae0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:54.141786 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65dadf7-039f-477d-abc7-d9e3cb98bae0 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:54.178005 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1153/2278] 10.222.0.89 () {60 vars in 1144 bytes} [Thu Jul 28 17:46:54 2022] POST /identity/v3/users => generated 299 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:46:54.179734 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65dadf7-039f-477d-abc7-d9e3cb98bae0 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:54.182140 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d9994e-dfe9-46c1-8934-75df04f25d72 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96d512d0c84240918dae6c4a3e8b7b79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:54.182969 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d9994e-dfe9-46c1-8934-75df04f25d72 service cinder] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:54.183378 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d9994e-dfe9-46c1-8934-75df04f25d72 service cinder] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:54.183656 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d9994e-dfe9-46c1-8934-75df04f25d72 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:54.194294 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65dadf7-039f-477d-abc7-d9e3cb98bae0 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:54.200380 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3f15d3-3b77-49d7-93e9-71f8ab72cb0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:54.201392 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3f15d3-3b77-49d7-93e9-71f8ab72cb0b admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:54.202347 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3f15d3-3b77-49d7-93e9-71f8ab72cb0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:54.203264 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3f15d3-3b77-49d7-93e9-71f8ab72cb0b admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:54.204445 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3f15d3-3b77-49d7-93e9-71f8ab72cb0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:54.206262 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3f15d3-3b77-49d7-93e9-71f8ab72cb0b admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:54.249782 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1154/2279] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:46:54 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:46:54.250092 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:46:54.251868 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3f15d3-3b77-49d7-93e9-71f8ab72cb0b admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:54.265891 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3f15d3-3b77-49d7-93e9-71f8ab72cb0b admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:54.270594 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d9994e-dfe9-46c1-8934-75df04f25d72 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:54.272574 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d9994e-dfe9-46c1-8934-75df04f25d72 service cinder] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:54.272786 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4389d4-84b7-4466-8069-b75804ef68f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:54.273990 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4389d4-84b7-4466-8069-b75804ef68f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:54.273990 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4389d4-84b7-4466-8069-b75804ef68f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:54.274193 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4389d4-84b7-4466-8069-b75804ef68f2 admin admin] PATH_INFO: `/v3/projects/c076770f4fd84ca2980c2bfa31c7baa4/users/726508c0e63347268bac205aca959f61/roles/73e457231a824ea98c28d22610be640c` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:54.290268 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:46:54.295334 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4389d4-84b7-4466-8069-b75804ef68f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c076770f4fd84ca2980c2bfa31c7baa4, user_id=726508c0e63347268bac205aca959f61, role_id=73e457231a824ea98c28d22610be640c)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:54.298374 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4389d4-84b7-4466-8069-b75804ef68f2 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:54.299152 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1126/2280] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:46:53 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:46:54.328884 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1155/2281] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:46:54 2022] PUT /identity/v3/projects/c076770f4fd84ca2980c2bfa31c7baa4/users/726508c0e63347268bac205aca959f61/roles/73e457231a824ea98c28d22610be640c => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 17:46:54.330146 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d9994e-dfe9-46c1-8934-75df04f25d72 service cinder] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:54.342437 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d9994e-dfe9-46c1-8934-75df04f25d72 service cinder] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:54.381152 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4389d4-84b7-4466-8069-b75804ef68f2 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:54.424380 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31906aed-c3ca-498b-be05-1342ae946f5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:54.425643 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31906aed-c3ca-498b-be05-1342ae946f5a admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:54.426073 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31906aed-c3ca-498b-be05-1342ae946f5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:54.426487 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31906aed-c3ca-498b-be05-1342ae946f5a admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:54.427663 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31906aed-c3ca-498b-be05-1342ae946f5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:54.429712 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31906aed-c3ca-498b-be05-1342ae946f5a admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:54.445433 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1127/2282] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:46:54 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:46:54.446928 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31906aed-c3ca-498b-be05-1342ae946f5a admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:54.460756 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31906aed-c3ca-498b-be05-1342ae946f5a admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:54.466019 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0f350b-a026-46fb-a61c-40ebc4a270a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:54.466718 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0f350b-a026-46fb-a61c-40ebc4a270a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:54.466993 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0f350b-a026-46fb-a61c-40ebc4a270a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:54.467262 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0f350b-a026-46fb-a61c-40ebc4a270a1 admin admin] PATH_INFO: `/v3/projects/c076770f4fd84ca2980c2bfa31c7baa4/users/726508c0e63347268bac205aca959f61/roles/73e457231a824ea98c28d22610be640c` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:54.470609 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0f350b-a026-46fb-a61c-40ebc4a270a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c076770f4fd84ca2980c2bfa31c7baa4, user_id=726508c0e63347268bac205aca959f61, role_id=73e457231a824ea98c28d22610be640c)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:54.473335 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0f350b-a026-46fb-a61c-40ebc4a270a1 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:54.489628 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1128/2283] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:46:54 2022] PUT /identity/v3/projects/c076770f4fd84ca2980c2bfa31c7baa4/users/726508c0e63347268bac205aca959f61/roles/73e457231a824ea98c28d22610be640c => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:46:54.490617 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4389d4-84b7-4466-8069-b75804ef68f2 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:54.501388 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcf0e08-c786-4795-93dd-329b2fceb630 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:54.501682 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcf0e08-c786-4795-93dd-329b2fceb630 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:54.501964 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcf0e08-c786-4795-93dd-329b2fceb630 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:54.534393 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-9fcf0e08-c786-4795-93dd-329b2fceb630 None None] MFA Rules not processed for user `726508c0e63347268bac205aca959f61`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:46:54.535875 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-9fcf0e08-c786-4795-93dd-329b2fceb630 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:46:54.556453 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d152004-27af-4b1d-9eec-43f0f503707e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96d512d0c84240918dae6c4a3e8b7b79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:54.557231 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d152004-27af-4b1d-9eec-43f0f503707e service cinder] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:54.557653 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d152004-27af-4b1d-9eec-43f0f503707e service cinder] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:54.558038 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d152004-27af-4b1d-9eec-43f0f503707e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:54.622452 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1129/2284] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:46:54 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:46:54.630586 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d152004-27af-4b1d-9eec-43f0f503707e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:54.632089 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d152004-27af-4b1d-9eec-43f0f503707e service cinder] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:54.642906 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1156/2285] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:46:54 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 17:46:54.670336 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcf0e08-c786-4795-93dd-329b2fceb630 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:54.681241 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcf0e08-c786-4795-93dd-329b2fceb630 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:54.685960 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a585065-c580-40e9-bb9d-8691cf28d821 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96d512d0c84240918dae6c4a3e8b7b79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:54.687458 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a585065-c580-40e9-bb9d-8691cf28d821 service cinder] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:54.687970 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a585065-c580-40e9-bb9d-8691cf28d821 service cinder] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:54.688461 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a585065-c580-40e9-bb9d-8691cf28d821 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:54.753985 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:46:54.759194 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a585065-c580-40e9-bb9d-8691cf28d821 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:54.760906 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a585065-c580-40e9-bb9d-8691cf28d821 service cinder] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:54.771379 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1130/2286] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:46:54 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:46:55.014240 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d152004-27af-4b1d-9eec-43f0f503707e service cinder] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:55.111681 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d152004-27af-4b1d-9eec-43f0f503707e service cinder] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:55.117040 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd0389f-1b19-4fa3-aa9c-a45ae9b572ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:55.117736 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd0389f-1b19-4fa3-aa9c-a45ae9b572ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:55.118020 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd0389f-1b19-4fa3-aa9c-a45ae9b572ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:55.118300 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd0389f-1b19-4fa3-aa9c-a45ae9b572ca admin admin] PATH_INFO: `/v3/users/905c91e9aabb4d1e9f2c0a68f9c1870d` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:55.119678 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd0389f-1b19-4fa3-aa9c-a45ae9b572ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=905c91e9aabb4d1e9f2c0a68f9c1870d)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:55.121671 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd0389f-1b19-4fa3-aa9c-a45ae9b572ca admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:55.123540 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:46:55.200230 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-8cd0389f-1b19-4fa3-aa9c-a45ae9b572ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '905c91e9aabb4d1e9f2c0a68f9c1870d'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:55.209708 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-8cd0389f-1b19-4fa3-aa9c-a45ae9b572ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '905c91e9aabb4d1e9f2c0a68f9c1870d'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:55.211033 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-8cd0389f-1b19-4fa3-aa9c-a45ae9b572ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '905c91e9aabb4d1e9f2c0a68f9c1870d'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:55.225937 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-8cd0389f-1b19-4fa3-aa9c-a45ae9b572ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '905c91e9aabb4d1e9f2c0a68f9c1870d'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:55.243957 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-8cd0389f-1b19-4fa3-aa9c-a45ae9b572ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '905c91e9aabb4d1e9f2c0a68f9c1870d'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:55.252475 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1157/2287] 10.222.0.89 () {58 vars in 1224 bytes} [Thu Jul 28 17:46:55 2022] DELETE /identity/v3/users/905c91e9aabb4d1e9f2c0a68f9c1870d => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 17:46:55.257388 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5f24f5-4a43-4d7c-96f6-42d6834bd7fb None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:55.257851 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5f24f5-4a43-4d7c-96f6-42d6834bd7fb None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:55.258125 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5f24f5-4a43-4d7c-96f6-42d6834bd7fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:55.283191 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-db5f24f5-4a43-4d7c-96f6-42d6834bd7fb None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:46:55.284602 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-db5f24f5-4a43-4d7c-96f6-42d6834bd7fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:46:55.361544 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1131/2288] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:46:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:46:55.368516 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd0389f-1b19-4fa3-aa9c-a45ae9b572ca admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:55.370259 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-8cd0389f-1b19-4fa3-aa9c-a45ae9b572ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:46:55.450303 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd0389f-1b19-4fa3-aa9c-a45ae9b572ca admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:55.455221 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16206bab-d318-4a17-ae4c-534370f036a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76817cd541294331b0023069092e8e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:55.456163 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16206bab-d318-4a17-ae4c-534370f036a8 service neutron] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:55.456437 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16206bab-d318-4a17-ae4c-534370f036a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:55.456719 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16206bab-d318-4a17-ae4c-534370f036a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:55.473887 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16206bab-d318-4a17-ae4c-534370f036a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:55.475369 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16206bab-d318-4a17-ae4c-534370f036a8 service neutron] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:55.487023 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1158/2289] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:46:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:46:55.901587 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5f24f5-4a43-4d7c-96f6-42d6834bd7fb None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:55.903111 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-db5f24f5-4a43-4d7c-96f6-42d6834bd7fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:46:55.916394 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5f24f5-4a43-4d7c-96f6-42d6834bd7fb None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:55.921663 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebb6029-8222-46e4-8b18-a3c0e0113b34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:55.922322 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebb6029-8222-46e4-8b18-a3c0e0113b34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:55.922751 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebb6029-8222-46e4-8b18-a3c0e0113b34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:55.923034 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebb6029-8222-46e4-8b18-a3c0e0113b34 admin admin] PATH_INFO: `/v3/projects/d9cbb5973aed44eab8e99ed8aceb1241` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:55.924233 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebb6029-8222-46e4-8b18-a3c0e0113b34 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9cbb5973aed44eab8e99ed8aceb1241)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:55.926187 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebb6029-8222-46e4-8b18-a3c0e0113b34 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:55.982210 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-6ebb6029-8222-46e4-8b18-a3c0e0113b34 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9cbb5973aed44eab8e99ed8aceb1241', 'request_id': 'req-6ebb6029-8222-46e4-8b18-a3c0e0113b34'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:55.988704 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-6ebb6029-8222-46e4-8b18-a3c0e0113b34 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9cbb5973aed44eab8e99ed8aceb1241', 'request_id': 'req-6ebb6029-8222-46e4-8b18-a3c0e0113b34'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:55.994135 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-6ebb6029-8222-46e4-8b18-a3c0e0113b34 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9cbb5973aed44eab8e99ed8aceb1241', 'request_id': 'req-6ebb6029-8222-46e4-8b18-a3c0e0113b34'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:56.007671 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-6ebb6029-8222-46e4-8b18-a3c0e0113b34 admin admin] The token cache is being invalidate because project d9cbb5973aed44eab8e99ed8aceb1241 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80325) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 17:46:56.007966 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-6ebb6029-8222-46e4-8b18-a3c0e0113b34 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:56.008610 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-6ebb6029-8222-46e4-8b18-a3c0e0113b34 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:56.010162 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1132/2290] 10.222.0.89 () {58 vars in 1233 bytes} [Thu Jul 28 17:46:55 2022] DELETE /identity/v3/projects/d9cbb5973aed44eab8e99ed8aceb1241 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 17:46:56.015179 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16206bab-d318-4a17-ae4c-534370f036a8 service neutron] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:56.025061 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16206bab-d318-4a17-ae4c-534370f036a8 service neutron] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:56.082944 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923575c0-dd58-467a-b6e1-fba4d0289d78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:56.083591 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923575c0-dd58-467a-b6e1-fba4d0289d78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:56.083877 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923575c0-dd58-467a-b6e1-fba4d0289d78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:56.084155 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923575c0-dd58-467a-b6e1-fba4d0289d78 admin admin] PATH_INFO: `/v3/users/937d966df5be4fb38bae46d127396edb` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:56.085467 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923575c0-dd58-467a-b6e1-fba4d0289d78 admin admin] RBAC: Authorizing `identity:delete_user(user_id=937d966df5be4fb38bae46d127396edb)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:56.087388 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923575c0-dd58-467a-b6e1-fba4d0289d78 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:56.113702 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:46:56.151078 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-923575c0-dd58-467a-b6e1-fba4d0289d78 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '937d966df5be4fb38bae46d127396edb'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:56.159503 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-923575c0-dd58-467a-b6e1-fba4d0289d78 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '937d966df5be4fb38bae46d127396edb'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:56.160762 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-923575c0-dd58-467a-b6e1-fba4d0289d78 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '937d966df5be4fb38bae46d127396edb'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:56.170364 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-923575c0-dd58-467a-b6e1-fba4d0289d78 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '937d966df5be4fb38bae46d127396edb'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:56.192963 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-923575c0-dd58-467a-b6e1-fba4d0289d78 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '937d966df5be4fb38bae46d127396edb'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:56.200852 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1159/2291] 10.222.0.89 () {58 vars in 1224 bytes} [Thu Jul 28 17:46:56 2022] DELETE /identity/v3/users/937d966df5be4fb38bae46d127396edb => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:46:56.504306 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebb6029-8222-46e4-8b18-a3c0e0113b34 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:56.505995 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-6ebb6029-8222-46e4-8b18-a3c0e0113b34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:46:56.564316 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923575c0-dd58-467a-b6e1-fba4d0289d78 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:56.566623 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-923575c0-dd58-467a-b6e1-fba4d0289d78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:46:56.591649 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebb6029-8222-46e4-8b18-a3c0e0113b34 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:56.633147 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:46:56.664938 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9e7601-cf6c-4ec3-a79b-1e0fceca5d69 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96d512d0c84240918dae6c4a3e8b7b79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:56.665514 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9e7601-cf6c-4ec3-a79b-1e0fceca5d69 service cinder] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:56.665793 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9e7601-cf6c-4ec3-a79b-1e0fceca5d69 service cinder] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:56.666164 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9e7601-cf6c-4ec3-a79b-1e0fceca5d69 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:56.668074 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-7f9e7601-cf6c-4ec3-a79b-1e0fceca5d69 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:46:56.717187 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923575c0-dd58-467a-b6e1-fba4d0289d78 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:56.722843 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c639f23b-8c36-4a33-ae37-e47f24b64a1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:56.723503 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c639f23b-8c36-4a33-ae37-e47f24b64a1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:56.723789 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c639f23b-8c36-4a33-ae37-e47f24b64a1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:56.724064 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c639f23b-8c36-4a33-ae37-e47f24b64a1e admin admin] PATH_INFO: `/v3/projects/d9dd7a97cfe540a38dccbad57810540d` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:56.725291 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c639f23b-8c36-4a33-ae37-e47f24b64a1e admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9dd7a97cfe540a38dccbad57810540d)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:56.727242 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c639f23b-8c36-4a33-ae37-e47f24b64a1e admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:56.753876 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9e7601-cf6c-4ec3-a79b-1e0fceca5d69 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:56.755319 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9e7601-cf6c-4ec3-a79b-1e0fceca5d69 service cinder] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:56.764881 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1133/2292] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:46:56 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:46:56.778403 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-c639f23b-8c36-4a33-ae37-e47f24b64a1e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9dd7a97cfe540a38dccbad57810540d', 'request_id': 'req-c639f23b-8c36-4a33-ae37-e47f24b64a1e'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:56.784750 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-c639f23b-8c36-4a33-ae37-e47f24b64a1e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9dd7a97cfe540a38dccbad57810540d', 'request_id': 'req-c639f23b-8c36-4a33-ae37-e47f24b64a1e'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:56.792159 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-c639f23b-8c36-4a33-ae37-e47f24b64a1e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9dd7a97cfe540a38dccbad57810540d', 'request_id': 'req-c639f23b-8c36-4a33-ae37-e47f24b64a1e'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:56.807914 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-c639f23b-8c36-4a33-ae37-e47f24b64a1e admin admin] The token cache is being invalidate because project d9dd7a97cfe540a38dccbad57810540d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80324) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 17:46:56.813525 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-c639f23b-8c36-4a33-ae37-e47f24b64a1e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:56.814229 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-c639f23b-8c36-4a33-ae37-e47f24b64a1e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:46:56.815880 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1160/2293] 10.222.0.89 () {58 vars in 1233 bytes} [Thu Jul 28 17:46:56 2022] DELETE /identity/v3/projects/d9dd7a97cfe540a38dccbad57810540d => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:46:56.820829 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f33f5a6-fe07-490e-a745-d2980d1bafba None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:56.821198 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f33f5a6-fe07-490e-a745-d2980d1bafba None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:56.821547 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f33f5a6-fe07-490e-a745-d2980d1bafba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:56.844611 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-6f33f5a6-fe07-490e-a745-d2980d1bafba None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:46:56.846477 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-6f33f5a6-fe07-490e-a745-d2980d1bafba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:46:56.961309 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1134/2294] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:46:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:46:56.968271 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a32347-a17b-4e8e-8013-4280270be405 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:56.968688 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a32347-a17b-4e8e-8013-4280270be405 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:56.969109 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a32347-a17b-4e8e-8013-4280270be405 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:56.992552 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-64a32347-a17b-4e8e-8013-4280270be405 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:46:56.993907 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-64a32347-a17b-4e8e-8013-4280270be405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:46:57.002669 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1161/2295] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:46:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 17:46:57.007347 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f33f5a6-fe07-490e-a745-d2980d1bafba None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:57.019643 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f33f5a6-fe07-490e-a745-d2980d1bafba None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:57.026402 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7417c24e-a2e0-47c2-b2e9-6a4a99768861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:57.027111 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7417c24e-a2e0-47c2-b2e9-6a4a99768861 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:57.027413 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7417c24e-a2e0-47c2-b2e9-6a4a99768861 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:57.027670 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7417c24e-a2e0-47c2-b2e9-6a4a99768861 admin admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:57.028394 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7417c24e-a2e0-47c2-b2e9-6a4a99768861 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:46:57.028699 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7417c24e-a2e0-47c2-b2e9-6a4a99768861 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:57.030570 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7417c24e-a2e0-47c2-b2e9-6a4a99768861 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:57.043837 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1135/2296] 10.222.0.89 () {58 vars in 1153 bytes} [Thu Jul 28 17:46:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:46:57.048783 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a32347-a17b-4e8e-8013-4280270be405 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:57.058222 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a32347-a17b-4e8e-8013-4280270be405 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:57.063068 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54b3101-d56b-479a-b5a2-b64503e8ee6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:57.063690 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54b3101-d56b-479a-b5a2-b64503e8ee6f admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:57.063947 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54b3101-d56b-479a-b5a2-b64503e8ee6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:57.064220 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54b3101-d56b-479a-b5a2-b64503e8ee6f admin admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:57.064955 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54b3101-d56b-479a-b5a2-b64503e8ee6f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:57.066895 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54b3101-d56b-479a-b5a2-b64503e8ee6f admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:57.097079 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1162/2297] 10.222.0.89 () {60 vars in 1153 bytes} [Thu Jul 28 17:46:57 2022] POST /identity/v3/projects => generated 377 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:46:57.101052 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7417c24e-a2e0-47c2-b2e9-6a4a99768861 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:57.111196 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7417c24e-a2e0-47c2-b2e9-6a4a99768861 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:57.184687 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1681b61e-65f1-4048-bbcb-1e489bada3fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:57.185554 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1681b61e-65f1-4048-bbcb-1e489bada3fb admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:57.185841 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1681b61e-65f1-4048-bbcb-1e489bada3fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:57.186144 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1681b61e-65f1-4048-bbcb-1e489bada3fb admin admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:57.187408 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1681b61e-65f1-4048-bbcb-1e489bada3fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:57.189399 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1681b61e-65f1-4048-bbcb-1e489bada3fb admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:57.216119 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1136/2298] 10.222.0.89 () {60 vars in 1144 bytes} [Thu Jul 28 17:46:57 2022] POST /identity/v3/users => generated 289 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 17:46:57.218304 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54b3101-d56b-479a-b5a2-b64503e8ee6f admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:57.225231 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:46:57.231124 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54b3101-d56b-479a-b5a2-b64503e8ee6f admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:57.238044 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32f178c-7814-4bd6-bc84-8ab2c1f2ef2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:57.238815 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32f178c-7814-4bd6-bc84-8ab2c1f2ef2e admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:57.239139 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32f178c-7814-4bd6-bc84-8ab2c1f2ef2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:57.240087 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32f178c-7814-4bd6-bc84-8ab2c1f2ef2e admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:57.240851 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32f178c-7814-4bd6-bc84-8ab2c1f2ef2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:57.242955 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32f178c-7814-4bd6-bc84-8ab2c1f2ef2e admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:57.256639 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1163/2299] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:46:57 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:46:57.261243 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1681b61e-65f1-4048-bbcb-1e489bada3fb admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:57.268442 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:46:57.273440 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1681b61e-65f1-4048-bbcb-1e489bada3fb admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:57.279709 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f26719e-a4af-4b64-8162-8ecb3664d826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:57.280220 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f26719e-a4af-4b64-8162-8ecb3664d826 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:57.281260 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f26719e-a4af-4b64-8162-8ecb3664d826 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:57.281575 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f26719e-a4af-4b64-8162-8ecb3664d826 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:57.282323 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f26719e-a4af-4b64-8162-8ecb3664d826 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:57.283983 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f26719e-a4af-4b64-8162-8ecb3664d826 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:57.296120 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1137/2300] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:46:57 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:46:57.302772 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32f178c-7814-4bd6-bc84-8ab2c1f2ef2e admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:57.313324 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32f178c-7814-4bd6-bc84-8ab2c1f2ef2e admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:57.318383 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704ded4b-193b-4348-8c30-4a8d539fdfba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:57.319059 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704ded4b-193b-4348-8c30-4a8d539fdfba admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:57.319732 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704ded4b-193b-4348-8c30-4a8d539fdfba admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:57.319909 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704ded4b-193b-4348-8c30-4a8d539fdfba admin admin] PATH_INFO: `/v3/projects/0cc8862d58e44881b784770c058f1ad2/users/1ca38f2b90f043dbbb3fa27fa743c6bf/roles/11f8731756fe4caca13efa61b71b11c5` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:57.341032 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704ded4b-193b-4348-8c30-4a8d539fdfba admin admin] RBAC: Authorizing `identity:create_grant(project_id=0cc8862d58e44881b784770c058f1ad2, user_id=1ca38f2b90f043dbbb3fa27fa743c6bf, role_id=11f8731756fe4caca13efa61b71b11c5)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:57.345031 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704ded4b-193b-4348-8c30-4a8d539fdfba admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:57.361932 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1164/2301] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:46:57 2022] PUT /identity/v3/projects/0cc8862d58e44881b784770c058f1ad2/users/1ca38f2b90f043dbbb3fa27fa743c6bf/roles/11f8731756fe4caca13efa61b71b11c5 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 17:46:57.433094 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd552b9-bf44-40ce-b9b2-ee20c577354a None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:57.433420 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd552b9-bf44-40ce-b9b2-ee20c577354a None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:57.433788 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd552b9-bf44-40ce-b9b2-ee20c577354a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:57.451725 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704ded4b-193b-4348-8c30-4a8d539fdfba admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:57.454493 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-704ded4b-193b-4348-8c30-4a8d539fdfba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:46:57.464484 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-ccd552b9-bf44-40ce-b9b2-ee20c577354a None None] MFA Rules not processed for user `1ca38f2b90f043dbbb3fa27fa743c6bf`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:46:57.465748 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-ccd552b9-bf44-40ce-b9b2-ee20c577354a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:46:57.528486 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704ded4b-193b-4348-8c30-4a8d539fdfba admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:57.532894 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708e42a9-b722-4467-9651-fad853edd251 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96d512d0c84240918dae6c4a3e8b7b79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:57.533611 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708e42a9-b722-4467-9651-fad853edd251 service cinder] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:57.533959 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708e42a9-b722-4467-9651-fad853edd251 service cinder] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:57.534174 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708e42a9-b722-4467-9651-fad853edd251 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:57.536399 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-708e42a9-b722-4467-9651-fad853edd251 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:46:57.552065 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1138/2302] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:46:57 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:46:57.564896 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd552b9-bf44-40ce-b9b2-ee20c577354a None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:46:57.566434 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-ccd552b9-bf44-40ce-b9b2-ee20c577354a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:46:57.633012 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708e42a9-b722-4467-9651-fad853edd251 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:57.634493 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708e42a9-b722-4467-9651-fad853edd251 service cinder] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:57.645618 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1165/2303] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:46:57 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 17:46:57.662933 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd552b9-bf44-40ce-b9b2-ee20c577354a None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:46:57.666618 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd0f9ac-aed7-40ab-9720-017bf7fb00c7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1564db0771224a05b47d8e3cf778c147', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:46:57.667396 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd0f9ac-aed7-40ab-9720-017bf7fb00c7 service glance] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:46:57.667682 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd0f9ac-aed7-40ab-9720-017bf7fb00c7 service glance] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:46:57.667878 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd0f9ac-aed7-40ab-9720-017bf7fb00c7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:46:57.678117 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd0f9ac-aed7-40ab-9720-017bf7fb00c7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:46:57.678580 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd0f9ac-aed7-40ab-9720-017bf7fb00c7 service glance] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:46:57.688938 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1139/2304] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:46:57 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 17:47:01.469405 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3098fe98-2cff-43ec-a472-e530b78f5000 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:47:01.470264 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3098fe98-2cff-43ec-a472-e530b78f5000 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:47:01.470264 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3098fe98-2cff-43ec-a472-e530b78f5000 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:47:01.492245 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-3098fe98-2cff-43ec-a472-e530b78f5000 None None] MFA Rules not processed for user `c44387d13a694ee6bb47a13906003330`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:47:01.493613 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-3098fe98-2cff-43ec-a472-e530b78f5000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:47:01.554554 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1166/2305] 10.222.0.89 () {58 vars in 1009 bytes} [Thu Jul 28 17:47:01 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:47:01.601942 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd0f9ac-aed7-40ab-9720-017bf7fb00c7 service glance] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:47:01.603466 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-9cd0f9ac-aed7-40ab-9720-017bf7fb00c7 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:47:01.708597 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd0f9ac-aed7-40ab-9720-017bf7fb00c7 service glance] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:47:01.714353 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c04c6a1-2613-43ed-a2e7-621202367081 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c59473b01bf4433ba3b111114fc048b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:47:01.715458 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c04c6a1-2613-43ed-a2e7-621202367081 service swift] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:47:01.715869 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c04c6a1-2613-43ed-a2e7-621202367081 service swift] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:47:01.716242 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c04c6a1-2613-43ed-a2e7-621202367081 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:47:01.719234 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3098fe98-2cff-43ec-a472-e530b78f5000 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:47:01.721658 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-3098fe98-2cff-43ec-a472-e530b78f5000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:47:01.730331 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:47:01.735461 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c04c6a1-2613-43ed-a2e7-621202367081 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:47:01.738007 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c04c6a1-2613-43ed-a2e7-621202367081 service swift] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:47:01.759775 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:47:01.769299 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1140/2306] 10.222.0.89 () {60 vars in 1384 bytes} [Thu Jul 28 17:47:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 17:47:01.827415 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3098fe98-2cff-43ec-a472-e530b78f5000 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:47:01.837214 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cb2b26-616e-40d7-9b47-4c9f70d7ac3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:47:01.837705 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cb2b26-616e-40d7-9b47-4c9f70d7ac3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:47:01.837985 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cb2b26-616e-40d7-9b47-4c9f70d7ac3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:47:01.838260 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cb2b26-616e-40d7-9b47-4c9f70d7ac3d admin admin] PATH_INFO: `/v3/users/9ddb8f47fdd343e8bafb283a3370c2e9` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:47:01.839474 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cb2b26-616e-40d7-9b47-4c9f70d7ac3d admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ddb8f47fdd343e8bafb283a3370c2e9)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:47:01.841493 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cb2b26-616e-40d7-9b47-4c9f70d7ac3d admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:47:01.907863 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-21cb2b26-616e-40d7-9b47-4c9f70d7ac3d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ddb8f47fdd343e8bafb283a3370c2e9'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:47:01.916361 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-21cb2b26-616e-40d7-9b47-4c9f70d7ac3d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ddb8f47fdd343e8bafb283a3370c2e9'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:47:01.917089 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-21cb2b26-616e-40d7-9b47-4c9f70d7ac3d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ddb8f47fdd343e8bafb283a3370c2e9'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:47:01.929369 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-21cb2b26-616e-40d7-9b47-4c9f70d7ac3d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ddb8f47fdd343e8bafb283a3370c2e9'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:47:01.944178 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-21cb2b26-616e-40d7-9b47-4c9f70d7ac3d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ddb8f47fdd343e8bafb283a3370c2e9'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:47:01.953191 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1167/2307] 10.222.0.89 () {58 vars in 1224 bytes} [Thu Jul 28 17:47:01 2022] DELETE /identity/v3/users/9ddb8f47fdd343e8bafb283a3370c2e9 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:47:01.958206 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcb2ca1-d8b7-41ba-973f-3aa202b33aee None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:47:01.958304 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcb2ca1-d8b7-41ba-973f-3aa202b33aee None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:47:01.958521 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcb2ca1-d8b7-41ba-973f-3aa202b33aee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:47:01.980476 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-afcb2ca1-d8b7-41ba-973f-3aa202b33aee None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:47:01.981730 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-afcb2ca1-d8b7-41ba-973f-3aa202b33aee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:47:02.081097 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1141/2308] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:47:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:47:02.090683 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cb2b26-616e-40d7-9b47-4c9f70d7ac3d admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:47:02.092389 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-21cb2b26-616e-40d7-9b47-4c9f70d7ac3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:47:02.180779 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cb2b26-616e-40d7-9b47-4c9f70d7ac3d admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:47:02.185629 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f9f840-34cb-4461-8cce-96aa46418005 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76817cd541294331b0023069092e8e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:47:02.186399 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f9f840-34cb-4461-8cce-96aa46418005 service neutron] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:47:02.186681 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f9f840-34cb-4461-8cce-96aa46418005 service neutron] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:47:02.186955 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f9f840-34cb-4461-8cce-96aa46418005 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:47:02.201692 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f9f840-34cb-4461-8cce-96aa46418005 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:47:02.203385 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f9f840-34cb-4461-8cce-96aa46418005 service neutron] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:47:02.214203 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1168/2309] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:47:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:47:02.506803 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6baa2f-39e1-44cd-9d1c-8ca5cba3cb85 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:47:02.507469 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6baa2f-39e1-44cd-9d1c-8ca5cba3cb85 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:47:02.507846 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6baa2f-39e1-44cd-9d1c-8ca5cba3cb85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:47:02.533413 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-8c6baa2f-39e1-44cd-9d1c-8ca5cba3cb85 None None] MFA Rules not processed for user `c44387d13a694ee6bb47a13906003330`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:47:02.534837 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-8c6baa2f-39e1-44cd-9d1c-8ca5cba3cb85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:47:02.619538 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f9f840-34cb-4461-8cce-96aa46418005 service neutron] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:47:02.621799 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-b3f9f840-34cb-4461-8cce-96aa46418005 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:47:02.641546 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f9f840-34cb-4461-8cce-96aa46418005 service neutron] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:47:02.646524 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1142/2310] 10.222.0.89 () {58 vars in 967 bytes} [Thu Jul 28 17:47:02 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:47:02.649863 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd249b6-c56b-4cd5-b60e-9681940223f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:47:02.650527 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd249b6-c56b-4cd5-b60e-9681940223f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:47:02.650814 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd249b6-c56b-4cd5-b60e-9681940223f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:47:02.651091 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd249b6-c56b-4cd5-b60e-9681940223f6 admin admin] PATH_INFO: `/v3/projects/dd646d06c265400aa0e578563e4e7cb3` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:47:02.653997 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd249b6-c56b-4cd5-b60e-9681940223f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd646d06c265400aa0e578563e4e7cb3)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:47:02.659605 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd249b6-c56b-4cd5-b60e-9681940223f6 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:47:02.663492 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6baa2f-39e1-44cd-9d1c-8ca5cba3cb85 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:47:02.666060 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-8c6baa2f-39e1-44cd-9d1c-8ca5cba3cb85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:47:02.730052 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:47:02.743041 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-acd249b6-c56b-4cd5-b60e-9681940223f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd646d06c265400aa0e578563e4e7cb3', 'request_id': 'req-acd249b6-c56b-4cd5-b60e-9681940223f6'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:47:02.752876 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-acd249b6-c56b-4cd5-b60e-9681940223f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd646d06c265400aa0e578563e4e7cb3', 'request_id': 'req-acd249b6-c56b-4cd5-b60e-9681940223f6'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:47:02.761679 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-acd249b6-c56b-4cd5-b60e-9681940223f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd646d06c265400aa0e578563e4e7cb3', 'request_id': 'req-acd249b6-c56b-4cd5-b60e-9681940223f6'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:47:02.765664 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6baa2f-39e1-44cd-9d1c-8ca5cba3cb85 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:47:02.790634 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-acd249b6-c56b-4cd5-b60e-9681940223f6 admin admin] The token cache is being invalidate because project dd646d06c265400aa0e578563e4e7cb3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80324) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 17:47:02.791127 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-acd249b6-c56b-4cd5-b60e-9681940223f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:47:02.791803 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-acd249b6-c56b-4cd5-b60e-9681940223f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:47:02.793418 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1169/2311] 10.222.0.89 () {58 vars in 1233 bytes} [Thu Jul 28 17:47:02 2022] DELETE /identity/v3/projects/dd646d06c265400aa0e578563e4e7cb3 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:47:02.797979 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd249b6-c56b-4cd5-b60e-9681940223f6 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:47:02.799549 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-acd249b6-c56b-4cd5-b60e-9681940223f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:47:02.834832 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:47:02.854531 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86551a81-6cfd-4c46-822d-22a011bbd23b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c59473b01bf4433ba3b111114fc048b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:47:02.855202 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86551a81-6cfd-4c46-822d-22a011bbd23b service swift] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:47:02.855485 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86551a81-6cfd-4c46-822d-22a011bbd23b service swift] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:47:02.855759 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86551a81-6cfd-4c46-822d-22a011bbd23b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:47:02.857661 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-86551a81-6cfd-4c46-822d-22a011bbd23b service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:47:02.899539 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd249b6-c56b-4cd5-b60e-9681940223f6 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:47:02.907819 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98356364-b251-4261-b85c-19c08807817f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:47:02.909338 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98356364-b251-4261-b85c-19c08807817f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:47:02.909883 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98356364-b251-4261-b85c-19c08807817f admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:47:02.910318 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98356364-b251-4261-b85c-19c08807817f admin admin] PATH_INFO: `/v3/users/39649ddac168401c98df597bf2a934d9` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:47:02.912604 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98356364-b251-4261-b85c-19c08807817f admin admin] RBAC: Authorizing `identity:delete_user(user_id=39649ddac168401c98df597bf2a934d9)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:47:02.915623 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98356364-b251-4261-b85c-19c08807817f admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:47:02.956022 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86551a81-6cfd-4c46-822d-22a011bbd23b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:47:02.957689 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86551a81-6cfd-4c46-822d-22a011bbd23b service swift] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:47:02.981407 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1143/2312] 10.222.0.89 () {60 vars in 1384 bytes} [Thu Jul 28 17:47:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 17:47:03.022498 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-98356364-b251-4261-b85c-19c08807817f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '39649ddac168401c98df597bf2a934d9'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:47:03.032988 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-98356364-b251-4261-b85c-19c08807817f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '39649ddac168401c98df597bf2a934d9'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:47:03.034818 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-98356364-b251-4261-b85c-19c08807817f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '39649ddac168401c98df597bf2a934d9'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:47:03.046467 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-98356364-b251-4261-b85c-19c08807817f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '39649ddac168401c98df597bf2a934d9'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:47:03.064037 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-98356364-b251-4261-b85c-19c08807817f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '39649ddac168401c98df597bf2a934d9'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:47:03.071447 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1170/2313] 10.222.0.89 () {58 vars in 1224 bytes} [Thu Jul 28 17:47:02 2022] DELETE /identity/v3/users/39649ddac168401c98df597bf2a934d9 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:47:03.359434 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b46cf43-e18f-48d6-9120-4e4bd923578c None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:47:03.359939 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b46cf43-e18f-48d6-9120-4e4bd923578c None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:47:03.360216 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b46cf43-e18f-48d6-9120-4e4bd923578c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:47:03.394342 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-8b46cf43-e18f-48d6-9120-4e4bd923578c None None] MFA Rules not processed for user `c44387d13a694ee6bb47a13906003330`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:47:03.395604 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-8b46cf43-e18f-48d6-9120-4e4bd923578c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:47:03.496454 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1144/2314] 10.222.0.89 () {58 vars in 1009 bytes} [Thu Jul 28 17:47:03 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:47:03.511487 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98356364-b251-4261-b85c-19c08807817f admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:47:03.513477 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-98356364-b251-4261-b85c-19c08807817f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:47:03.574282 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b46cf43-e18f-48d6-9120-4e4bd923578c None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:47:03.576325 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-8b46cf43-e18f-48d6-9120-4e4bd923578c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:47:03.621083 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:47:03.648951 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98356364-b251-4261-b85c-19c08807817f admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:47:03.668773 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5695aedb-aa61-4849-8107-0914d34fa229 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c59473b01bf4433ba3b111114fc048b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:47:03.669448 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5695aedb-aa61-4849-8107-0914d34fa229 service swift] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:47:03.669733 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5695aedb-aa61-4849-8107-0914d34fa229 service swift] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:47:03.670010 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5695aedb-aa61-4849-8107-0914d34fa229 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:47:03.683770 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5695aedb-aa61-4849-8107-0914d34fa229 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:47:03.685484 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5695aedb-aa61-4849-8107-0914d34fa229 service swift] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:47:03.697315 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1171/2315] 10.222.0.89 () {60 vars in 1384 bytes} [Thu Jul 28 17:47:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 17:47:03.793340 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b46cf43-e18f-48d6-9120-4e4bd923578c None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:47:03.800880 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6d2a02-fc22-49e2-9044-6d7f086cf02e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:47:03.802028 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6d2a02-fc22-49e2-9044-6d7f086cf02e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:47:03.802540 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6d2a02-fc22-49e2-9044-6d7f086cf02e admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:47:03.803135 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6d2a02-fc22-49e2-9044-6d7f086cf02e admin admin] PATH_INFO: `/v3/projects/6f70e34bfec746159e123060fac97115` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:47:03.804757 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6d2a02-fc22-49e2-9044-6d7f086cf02e admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f70e34bfec746159e123060fac97115)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:47:03.807320 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6d2a02-fc22-49e2-9044-6d7f086cf02e admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:47:03.907176 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:47:03.927270 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-de6d2a02-fc22-49e2-9044-6d7f086cf02e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f70e34bfec746159e123060fac97115', 'request_id': 'req-de6d2a02-fc22-49e2-9044-6d7f086cf02e'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:47:03.942424 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-de6d2a02-fc22-49e2-9044-6d7f086cf02e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f70e34bfec746159e123060fac97115', 'request_id': 'req-de6d2a02-fc22-49e2-9044-6d7f086cf02e'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:47:03.949650 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-de6d2a02-fc22-49e2-9044-6d7f086cf02e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f70e34bfec746159e123060fac97115', 'request_id': 'req-de6d2a02-fc22-49e2-9044-6d7f086cf02e'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:47:03.974891 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-de6d2a02-fc22-49e2-9044-6d7f086cf02e admin admin] The token cache is being invalidate because project 6f70e34bfec746159e123060fac97115 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80325) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 17:47:03.975198 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-de6d2a02-fc22-49e2-9044-6d7f086cf02e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:47:03.977573 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-de6d2a02-fc22-49e2-9044-6d7f086cf02e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:47:03.979483 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1145/2316] 10.222.0.89 () {58 vars in 1233 bytes} [Thu Jul 28 17:47:03 2022] DELETE /identity/v3/projects/6f70e34bfec746159e123060fac97115 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 17:47:03.986091 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5695aedb-aa61-4849-8107-0914d34fa229 service swift] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:47:03.998453 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5695aedb-aa61-4849-8107-0914d34fa229 service swift] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:47:04.101419 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fa5044-27ea-4d0c-af8f-40a96786b805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:47:04.102480 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fa5044-27ea-4d0c-af8f-40a96786b805 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:47:04.103036 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fa5044-27ea-4d0c-af8f-40a96786b805 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:47:04.103521 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fa5044-27ea-4d0c-af8f-40a96786b805 admin admin] PATH_INFO: `/v3/users/726508c0e63347268bac205aca959f61` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:47:04.117505 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fa5044-27ea-4d0c-af8f-40a96786b805 admin admin] RBAC: Authorizing `identity:delete_user(user_id=726508c0e63347268bac205aca959f61)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:47:04.120006 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fa5044-27ea-4d0c-af8f-40a96786b805 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:47:04.212758 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-70fa5044-27ea-4d0c-af8f-40a96786b805 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '726508c0e63347268bac205aca959f61'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:47:04.223977 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df68efb-19ea-4508-92b3-c9d28e32e1e3 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:47:04.224594 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df68efb-19ea-4508-92b3-c9d28e32e1e3 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:47:04.225026 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df68efb-19ea-4508-92b3-c9d28e32e1e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:47:04.232126 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-70fa5044-27ea-4d0c-af8f-40a96786b805 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '726508c0e63347268bac205aca959f61'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:47:04.234690 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-70fa5044-27ea-4d0c-af8f-40a96786b805 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '726508c0e63347268bac205aca959f61'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:47:04.248336 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-2df68efb-19ea-4508-92b3-c9d28e32e1e3 None None] MFA Rules not processed for user `c44387d13a694ee6bb47a13906003330`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:47:04.251959 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-2df68efb-19ea-4508-92b3-c9d28e32e1e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:47:04.258387 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-70fa5044-27ea-4d0c-af8f-40a96786b805 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '726508c0e63347268bac205aca959f61'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:47:04.277368 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-70fa5044-27ea-4d0c-af8f-40a96786b805 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '726508c0e63347268bac205aca959f61'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:47:04.284925 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1172/2317] 10.222.0.89 () {58 vars in 1224 bytes} [Thu Jul 28 17:47:03 2022] DELETE /identity/v3/users/726508c0e63347268bac205aca959f61 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:47:04.338069 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1146/2318] 10.222.0.89 () {58 vars in 967 bytes} [Thu Jul 28 17:47:04 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 17:47:04.352853 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fa5044-27ea-4d0c-af8f-40a96786b805 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:47:04.354732 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-70fa5044-27ea-4d0c-af8f-40a96786b805 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:47:04.441402 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:47:04.461990 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fa5044-27ea-4d0c-af8f-40a96786b805 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:47:04.469962 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdd6ffc-8166-456a-84f3-684ebd8173a8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c59473b01bf4433ba3b111114fc048b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:47:04.470714 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdd6ffc-8166-456a-84f3-684ebd8173a8 service swift] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:47:04.471093 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdd6ffc-8166-456a-84f3-684ebd8173a8 service swift] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:47:04.471467 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdd6ffc-8166-456a-84f3-684ebd8173a8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:47:04.558868 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdd6ffc-8166-456a-84f3-684ebd8173a8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:47:04.560646 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdd6ffc-8166-456a-84f3-684ebd8173a8 service swift] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:47:04.575787 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1173/2319] 10.222.0.89 () {60 vars in 1384 bytes} [Thu Jul 28 17:47:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 17:47:04.778369 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df68efb-19ea-4508-92b3-c9d28e32e1e3 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:47:04.782959 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-2df68efb-19ea-4508-92b3-c9d28e32e1e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:47:04.811449 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df68efb-19ea-4508-92b3-c9d28e32e1e3 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:47:04.909727 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04e6ff4-b151-47fb-bd09-42c3c6d8aa01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:47:04.910555 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04e6ff4-b151-47fb-bd09-42c3c6d8aa01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:47:04.910942 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04e6ff4-b151-47fb-bd09-42c3c6d8aa01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:47:04.911388 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04e6ff4-b151-47fb-bd09-42c3c6d8aa01 admin admin] PATH_INFO: `/v3/projects/c076770f4fd84ca2980c2bfa31c7baa4` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:47:04.915306 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04e6ff4-b151-47fb-bd09-42c3c6d8aa01 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c076770f4fd84ca2980c2bfa31c7baa4)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:47:04.917010 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04e6ff4-b151-47fb-bd09-42c3c6d8aa01 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:47:04.982208 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-c04e6ff4-b151-47fb-bd09-42c3c6d8aa01 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c076770f4fd84ca2980c2bfa31c7baa4', 'request_id': 'req-c04e6ff4-b151-47fb-bd09-42c3c6d8aa01'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:47:04.991671 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-c04e6ff4-b151-47fb-bd09-42c3c6d8aa01 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c076770f4fd84ca2980c2bfa31c7baa4', 'request_id': 'req-c04e6ff4-b151-47fb-bd09-42c3c6d8aa01'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:47:04.997930 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-c04e6ff4-b151-47fb-bd09-42c3c6d8aa01 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c076770f4fd84ca2980c2bfa31c7baa4', 'request_id': 'req-c04e6ff4-b151-47fb-bd09-42c3c6d8aa01'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:47:05.002194 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:47:05.015238 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-c04e6ff4-b151-47fb-bd09-42c3c6d8aa01 admin admin] The token cache is being invalidate because project c076770f4fd84ca2980c2bfa31c7baa4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80325) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 17:47:05.015602 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-c04e6ff4-b151-47fb-bd09-42c3c6d8aa01 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:47:05.016312 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-c04e6ff4-b151-47fb-bd09-42c3c6d8aa01 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:47:05.017962 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1147/2320] 10.222.0.89 () {58 vars in 1233 bytes} [Thu Jul 28 17:47:04 2022] DELETE /identity/v3/projects/c076770f4fd84ca2980c2bfa31c7baa4 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:47:05.022995 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8fbc8b-e02d-4906-979b-f8a4d3e7fb46 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:47:05.023497 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8fbc8b-e02d-4906-979b-f8a4d3e7fb46 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:47:05.023752 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8fbc8b-e02d-4906-979b-f8a4d3e7fb46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:47:05.044049 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-bf8fbc8b-e02d-4906-979b-f8a4d3e7fb46 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:47:05.050873 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-bf8fbc8b-e02d-4906-979b-f8a4d3e7fb46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:47:05.114885 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1174/2321] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:47:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 17:47:05.122722 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd99c110-ece2-4c9e-ad7f-5eeb9a99f3e8 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:47:05.123007 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd99c110-ece2-4c9e-ad7f-5eeb9a99f3e8 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:47:05.123310 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd99c110-ece2-4c9e-ad7f-5eeb9a99f3e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:47:05.147729 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-cd99c110-ece2-4c9e-ad7f-5eeb9a99f3e8 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:47:05.149011 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-cd99c110-ece2-4c9e-ad7f-5eeb9a99f3e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:47:05.161644 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1148/2322] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:47:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 17:47:05.162903 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8fbc8b-e02d-4906-979b-f8a4d3e7fb46 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:47:05.173239 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8fbc8b-e02d-4906-979b-f8a4d3e7fb46 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:47:05.178214 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528e2540-0407-4097-b22f-bcb008977aa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:47:05.178823 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528e2540-0407-4097-b22f-bcb008977aa1 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:47:05.179102 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528e2540-0407-4097-b22f-bcb008977aa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:47:05.179373 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528e2540-0407-4097-b22f-bcb008977aa1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:47:05.180137 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528e2540-0407-4097-b22f-bcb008977aa1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:47:05.180418 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528e2540-0407-4097-b22f-bcb008977aa1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:47:05.183325 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528e2540-0407-4097-b22f-bcb008977aa1 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:47:05.197139 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1175/2323] 10.222.0.89 () {58 vars in 1153 bytes} [Thu Jul 28 17:47:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:47:05.201525 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd99c110-ece2-4c9e-ad7f-5eeb9a99f3e8 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:47:05.212164 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd99c110-ece2-4c9e-ad7f-5eeb9a99f3e8 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:47:05.217270 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa06cc3-f64e-4253-9336-cf7b1688aa3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:47:05.218052 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa06cc3-f64e-4253-9336-cf7b1688aa3c admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:47:05.218433 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa06cc3-f64e-4253-9336-cf7b1688aa3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:47:05.218962 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa06cc3-f64e-4253-9336-cf7b1688aa3c admin admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:47:05.220049 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa06cc3-f64e-4253-9336-cf7b1688aa3c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:47:05.222062 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa06cc3-f64e-4253-9336-cf7b1688aa3c admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:47:05.256192 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1149/2324] 10.222.0.89 () {60 vars in 1153 bytes} [Thu Jul 28 17:47:05 2022] POST /identity/v3/projects => generated 369 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:47:05.260230 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528e2540-0407-4097-b22f-bcb008977aa1 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:47:05.271525 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528e2540-0407-4097-b22f-bcb008977aa1 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:47:05.349888 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa7c5bb-f9c3-4830-80cb-07a9eb959188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:47:05.350498 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa7c5bb-f9c3-4830-80cb-07a9eb959188 admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:47:05.350799 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa7c5bb-f9c3-4830-80cb-07a9eb959188 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:47:05.351073 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa7c5bb-f9c3-4830-80cb-07a9eb959188 admin admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:47:05.351728 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa7c5bb-f9c3-4830-80cb-07a9eb959188 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:47:05.353662 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa7c5bb-f9c3-4830-80cb-07a9eb959188 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:47:05.380918 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1176/2325] 10.222.0.89 () {60 vars in 1144 bytes} [Thu Jul 28 17:47:05 2022] POST /identity/v3/users => generated 285 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:47:05.384925 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa06cc3-f64e-4253-9336-cf7b1688aa3c admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:47:05.398008 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa06cc3-f64e-4253-9336-cf7b1688aa3c admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:47:05.402866 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c56d9c0-01c6-4d43-8fa5-fe1f62b78cd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:47:05.403477 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c56d9c0-01c6-4d43-8fa5-fe1f62b78cd2 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:47:05.403755 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c56d9c0-01c6-4d43-8fa5-fe1f62b78cd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:47:05.404020 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c56d9c0-01c6-4d43-8fa5-fe1f62b78cd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:47:05.404706 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c56d9c0-01c6-4d43-8fa5-fe1f62b78cd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:47:05.407778 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c56d9c0-01c6-4d43-8fa5-fe1f62b78cd2 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:47:05.419535 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1150/2326] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:47:05 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:47:05.423810 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa7c5bb-f9c3-4830-80cb-07a9eb959188 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:47:05.433627 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa7c5bb-f9c3-4830-80cb-07a9eb959188 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:47:05.439827 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d21870-80a8-4a33-a704-c444a8e0b74f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:47:05.440536 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d21870-80a8-4a33-a704-c444a8e0b74f admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:47:05.440825 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d21870-80a8-4a33-a704-c444a8e0b74f admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:47:05.441101 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d21870-80a8-4a33-a704-c444a8e0b74f admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:47:05.441853 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d21870-80a8-4a33-a704-c444a8e0b74f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:47:05.443434 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d21870-80a8-4a33-a704-c444a8e0b74f admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:47:05.455832 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1177/2327] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:47:05 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:47:05.460123 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c56d9c0-01c6-4d43-8fa5-fe1f62b78cd2 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:47:05.473703 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c56d9c0-01c6-4d43-8fa5-fe1f62b78cd2 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:47:05.478466 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cbb732-647a-4e36-9b08-49c068b6ca49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:47:05.479134 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cbb732-647a-4e36-9b08-49c068b6ca49 admin admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:47:05.479397 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cbb732-647a-4e36-9b08-49c068b6ca49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:47:05.479673 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cbb732-647a-4e36-9b08-49c068b6ca49 admin admin] PATH_INFO: `/v3/projects/b4e1c233487d48c69cdbbd059a0c3108/users/88d61d95ab7a4e5c9413ee94fc936d34/roles/11f8731756fe4caca13efa61b71b11c5` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:47:05.495838 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cbb732-647a-4e36-9b08-49c068b6ca49 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4e1c233487d48c69cdbbd059a0c3108, user_id=88d61d95ab7a4e5c9413ee94fc936d34, role_id=11f8731756fe4caca13efa61b71b11c5)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:47:05.500116 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cbb732-647a-4e36-9b08-49c068b6ca49 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:47:05.515410 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1151/2328] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:47:05 2022] PUT /identity/v3/projects/b4e1c233487d48c69cdbbd059a0c3108/users/88d61d95ab7a4e5c9413ee94fc936d34/roles/11f8731756fe4caca13efa61b71b11c5 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 17:47:05.529629 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee94a10-01dc-4eae-99cd-9f8252ad7657 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:47:05.529629 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee94a10-01dc-4eae-99cd-9f8252ad7657 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:47:05.529629 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee94a10-01dc-4eae-99cd-9f8252ad7657 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:47:05.535854 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:47:05.561881 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-1ee94a10-01dc-4eae-99cd-9f8252ad7657 None None] MFA Rules not processed for user `88d61d95ab7a4e5c9413ee94fc936d34`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:47:05.563398 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-1ee94a10-01dc-4eae-99cd-9f8252ad7657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:47:05.652799 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1178/2329] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:47:05 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:47:05.666434 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cbb732-647a-4e36-9b08-49c068b6ca49 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:47:05.672496 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-10cbb732-647a-4e36-9b08-49c068b6ca49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:47:05.758200 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cbb732-647a-4e36-9b08-49c068b6ca49 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:47:05.766442 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025f571c-20f6-4e1d-aac7-d51b341e13d6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1564db0771224a05b47d8e3cf778c147', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:47:05.767117 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025f571c-20f6-4e1d-aac7-d51b341e13d6 service glance] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:47:05.767361 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025f571c-20f6-4e1d-aac7-d51b341e13d6 service glance] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:47:05.767642 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025f571c-20f6-4e1d-aac7-d51b341e13d6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:47:05.777826 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025f571c-20f6-4e1d-aac7-d51b341e13d6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:47:05.779350 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025f571c-20f6-4e1d-aac7-d51b341e13d6 service glance] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:47:05.796441 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1152/2330] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:47:05 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:47:06.901889 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee94a10-01dc-4eae-99cd-9f8252ad7657 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:47:06.903327 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-1ee94a10-01dc-4eae-99cd-9f8252ad7657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:47:06.975485 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee94a10-01dc-4eae-99cd-9f8252ad7657 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:47:06.980168 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3556bff-94dd-4974-8ccf-c7d12b650ebf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:47:06.980802 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3556bff-94dd-4974-8ccf-c7d12b650ebf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:47:06.981079 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3556bff-94dd-4974-8ccf-c7d12b650ebf admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:47:06.981374 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3556bff-94dd-4974-8ccf-c7d12b650ebf admin admin] PATH_INFO: `/v3/users/902d3de9354b4eefb3d99b6e91432c74` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:47:06.982542 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3556bff-94dd-4974-8ccf-c7d12b650ebf admin admin] RBAC: Authorizing `identity:delete_user(user_id=902d3de9354b4eefb3d99b6e91432c74)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:47:06.984564 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3556bff-94dd-4974-8ccf-c7d12b650ebf admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:47:07.032306 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-b3556bff-94dd-4974-8ccf-c7d12b650ebf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '902d3de9354b4eefb3d99b6e91432c74'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:47:07.039079 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-b3556bff-94dd-4974-8ccf-c7d12b650ebf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '902d3de9354b4eefb3d99b6e91432c74'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:47:07.039740 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-b3556bff-94dd-4974-8ccf-c7d12b650ebf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '902d3de9354b4eefb3d99b6e91432c74'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:47:07.047202 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-b3556bff-94dd-4974-8ccf-c7d12b650ebf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '902d3de9354b4eefb3d99b6e91432c74'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:47:07.060113 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-b3556bff-94dd-4974-8ccf-c7d12b650ebf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '902d3de9354b4eefb3d99b6e91432c74'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:47:07.067276 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1179/2331] 10.222.0.89 () {58 vars in 1224 bytes} [Thu Jul 28 17:47:06 2022] DELETE /identity/v3/users/902d3de9354b4eefb3d99b6e91432c74 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:47:07.075815 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ff75d4-de99-42bd-ba50-b795786ecebc None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:47:07.076317 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ff75d4-de99-42bd-ba50-b795786ecebc None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:47:07.076437 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ff75d4-de99-42bd-ba50-b795786ecebc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:47:07.100303 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-37ff75d4-de99-42bd-ba50-b795786ecebc None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:47:07.101569 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-37ff75d4-de99-42bd-ba50-b795786ecebc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:47:07.161639 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1153/2332] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:47:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:47:07.171384 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3556bff-94dd-4974-8ccf-c7d12b650ebf admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:47:07.172948 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-b3556bff-94dd-4974-8ccf-c7d12b650ebf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:47:07.186896 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:47:07.235910 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3556bff-94dd-4974-8ccf-c7d12b650ebf admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:47:07.239859 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bde793-99c0-46f3-b6e8-f7a85f1978d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76817cd541294331b0023069092e8e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:47:07.240484 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bde793-99c0-46f3-b6e8-f7a85f1978d6 service neutron] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:47:07.240767 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bde793-99c0-46f3-b6e8-f7a85f1978d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:47:07.241041 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bde793-99c0-46f3-b6e8-f7a85f1978d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:47:07.251673 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bde793-99c0-46f3-b6e8-f7a85f1978d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:47:07.253110 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bde793-99c0-46f3-b6e8-f7a85f1978d6 service neutron] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:47:07.264416 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1180/2333] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:47:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:47:07.442463 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ff75d4-de99-42bd-ba50-b795786ecebc None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:47:07.444019 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-37ff75d4-de99-42bd-ba50-b795786ecebc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:47:07.457777 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ff75d4-de99-42bd-ba50-b795786ecebc None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:47:07.462372 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9774348e-02b2-48fb-b686-550027464eb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:47:07.463092 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9774348e-02b2-48fb-b686-550027464eb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:47:07.463442 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9774348e-02b2-48fb-b686-550027464eb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:47:07.463781 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9774348e-02b2-48fb-b686-550027464eb1 admin admin] PATH_INFO: `/v3/projects/a83427bbf87f4dbe9e3533033c075c9e` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:47:07.465012 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9774348e-02b2-48fb-b686-550027464eb1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a83427bbf87f4dbe9e3533033c075c9e)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:47:07.467301 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9774348e-02b2-48fb-b686-550027464eb1 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:47:07.506996 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:47:07.522020 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-9774348e-02b2-48fb-b686-550027464eb1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a83427bbf87f4dbe9e3533033c075c9e', 'request_id': 'req-9774348e-02b2-48fb-b686-550027464eb1'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:47:07.528408 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-9774348e-02b2-48fb-b686-550027464eb1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a83427bbf87f4dbe9e3533033c075c9e', 'request_id': 'req-9774348e-02b2-48fb-b686-550027464eb1'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:47:07.535349 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-9774348e-02b2-48fb-b686-550027464eb1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a83427bbf87f4dbe9e3533033c075c9e', 'request_id': 'req-9774348e-02b2-48fb-b686-550027464eb1'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:47:07.550276 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-9774348e-02b2-48fb-b686-550027464eb1 admin admin] The token cache is being invalidate because project a83427bbf87f4dbe9e3533033c075c9e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80325) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 17:47:07.550597 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-9774348e-02b2-48fb-b686-550027464eb1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:47:07.551226 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-9774348e-02b2-48fb-b686-550027464eb1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:47:07.552992 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1154/2334] 10.222.0.89 () {58 vars in 1233 bytes} [Thu Jul 28 17:47:07 2022] DELETE /identity/v3/projects/a83427bbf87f4dbe9e3533033c075c9e => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:47:07.558042 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f07f79-27bf-4ed3-a0a7-83f0709aa25b None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:47:07.558333 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f07f79-27bf-4ed3-a0a7-83f0709aa25b None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:47:07.558638 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f07f79-27bf-4ed3-a0a7-83f0709aa25b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:47:07.583193 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-d0f07f79-27bf-4ed3-a0a7-83f0709aa25b None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:47:07.584768 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-d0f07f79-27bf-4ed3-a0a7-83f0709aa25b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:47:07.658186 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1181/2335] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:47:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 17:47:07.665792 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9bb8ae-7da3-49ef-adc8-e6a0246529bd None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:47:07.666268 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9bb8ae-7da3-49ef-adc8-e6a0246529bd None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:47:07.666540 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9bb8ae-7da3-49ef-adc8-e6a0246529bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:47:07.689936 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-aa9bb8ae-7da3-49ef-adc8-e6a0246529bd None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:47:07.689936 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-aa9bb8ae-7da3-49ef-adc8-e6a0246529bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:47:07.698382 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1155/2336] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:47:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:47:07.702982 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f07f79-27bf-4ed3-a0a7-83f0709aa25b None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:47:07.712980 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f07f79-27bf-4ed3-a0a7-83f0709aa25b None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:47:07.717742 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4b276f-2143-4b80-aef1-a4b59e380b4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:47:07.718353 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4b276f-2143-4b80-aef1-a4b59e380b4b admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:47:07.718671 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4b276f-2143-4b80-aef1-a4b59e380b4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:47:07.718951 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4b276f-2143-4b80-aef1-a4b59e380b4b admin admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:47:07.719650 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4b276f-2143-4b80-aef1-a4b59e380b4b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:47:07.719950 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4b276f-2143-4b80-aef1-a4b59e380b4b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:47:07.721659 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4b276f-2143-4b80-aef1-a4b59e380b4b admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:47:07.733780 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1182/2337] 10.222.0.89 () {58 vars in 1153 bytes} [Thu Jul 28 17:47:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:47:07.738281 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9bb8ae-7da3-49ef-adc8-e6a0246529bd None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:47:07.747471 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9bb8ae-7da3-49ef-adc8-e6a0246529bd None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:47:07.753725 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2287826-184d-4757-86fa-92d8cc366f21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:47:07.754354 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2287826-184d-4757-86fa-92d8cc366f21 admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:47:07.754642 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2287826-184d-4757-86fa-92d8cc366f21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:47:07.754915 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2287826-184d-4757-86fa-92d8cc366f21 admin admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:47:07.755608 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2287826-184d-4757-86fa-92d8cc366f21 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:47:07.757548 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2287826-184d-4757-86fa-92d8cc366f21 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:47:07.785275 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1156/2338] 10.222.0.89 () {60 vars in 1153 bytes} [Thu Jul 28 17:47:07 2022] POST /identity/v3/projects => generated 383 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:47:07.789533 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4b276f-2143-4b80-aef1-a4b59e380b4b admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:47:07.800434 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4b276f-2143-4b80-aef1-a4b59e380b4b admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:47:07.859877 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c68534-3fa9-4329-ae7e-65c4055b9f9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:47:07.860767 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c68534-3fa9-4329-ae7e-65c4055b9f9d admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:47:07.861346 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c68534-3fa9-4329-ae7e-65c4055b9f9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:47:07.861622 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c68534-3fa9-4329-ae7e-65c4055b9f9d admin admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:47:07.862437 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c68534-3fa9-4329-ae7e-65c4055b9f9d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:47:07.864508 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c68534-3fa9-4329-ae7e-65c4055b9f9d admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:47:07.897111 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1183/2339] 10.222.0.89 () {60 vars in 1144 bytes} [Thu Jul 28 17:47:07 2022] POST /identity/v3/users => generated 292 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:47:07.901218 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2287826-184d-4757-86fa-92d8cc366f21 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:47:07.912036 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2287826-184d-4757-86fa-92d8cc366f21 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:47:07.916746 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed228ae-d79d-4260-a0b0-4596e76fc637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:47:07.917546 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed228ae-d79d-4260-a0b0-4596e76fc637 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:47:07.917831 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed228ae-d79d-4260-a0b0-4596e76fc637 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:47:07.918103 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed228ae-d79d-4260-a0b0-4596e76fc637 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:47:07.918792 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed228ae-d79d-4260-a0b0-4596e76fc637 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:47:07.920415 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed228ae-d79d-4260-a0b0-4596e76fc637 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:47:07.933145 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1157/2340] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:47:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 17:47:07.935891 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c68534-3fa9-4329-ae7e-65c4055b9f9d admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:47:07.946209 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c68534-3fa9-4329-ae7e-65c4055b9f9d admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:47:07.950937 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76a32fc-c7dc-4a36-9d6f-45bc091523c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:47:07.951545 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76a32fc-c7dc-4a36-9d6f-45bc091523c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:47:07.951854 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76a32fc-c7dc-4a36-9d6f-45bc091523c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:47:07.952133 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76a32fc-c7dc-4a36-9d6f-45bc091523c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:47:07.952973 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76a32fc-c7dc-4a36-9d6f-45bc091523c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:47:07.954603 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76a32fc-c7dc-4a36-9d6f-45bc091523c0 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:47:07.967150 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1184/2341] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:47:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:47:07.972160 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed228ae-d79d-4260-a0b0-4596e76fc637 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:47:07.983470 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed228ae-d79d-4260-a0b0-4596e76fc637 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:47:07.988869 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac80c9c-7ad5-4ccb-acb0-9765bc180525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:47:07.989678 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac80c9c-7ad5-4ccb-acb0-9765bc180525 admin admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:47:07.989961 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac80c9c-7ad5-4ccb-acb0-9765bc180525 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:47:07.990239 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac80c9c-7ad5-4ccb-acb0-9765bc180525 admin admin] PATH_INFO: `/v3/projects/0513cb18c0cd43d5a4784f573bb382d4/users/ccc41bad96884148a0088cc8048a2316/roles/11f8731756fe4caca13efa61b71b11c5` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:47:08.007302 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac80c9c-7ad5-4ccb-acb0-9765bc180525 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0513cb18c0cd43d5a4784f573bb382d4, user_id=ccc41bad96884148a0088cc8048a2316, role_id=11f8731756fe4caca13efa61b71b11c5)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:47:08.010004 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac80c9c-7ad5-4ccb-acb0-9765bc180525 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:47:08.023775 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1158/2342] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:47:07 2022] PUT /identity/v3/projects/0513cb18c0cd43d5a4784f573bb382d4/users/ccc41bad96884148a0088cc8048a2316/roles/11f8731756fe4caca13efa61b71b11c5 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:47:08.028724 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d915a47-f26c-4cc1-a64c-75630cef2d31 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:47:08.029009 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d915a47-f26c-4cc1-a64c-75630cef2d31 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:47:08.029318 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d915a47-f26c-4cc1-a64c-75630cef2d31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:47:08.049845 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-9d915a47-f26c-4cc1-a64c-75630cef2d31 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:47:08.051068 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-9d915a47-f26c-4cc1-a64c-75630cef2d31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:47:08.067492 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:47:08.125858 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1185/2343] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:47:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:47:08.135215 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac80c9c-7ad5-4ccb-acb0-9765bc180525 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:47:08.137074 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-dac80c9c-7ad5-4ccb-acb0-9765bc180525 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:47:08.194704 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac80c9c-7ad5-4ccb-acb0-9765bc180525 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:47:08.198345 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b5ade4-c54b-4d7d-ba1c-8446236214b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76817cd541294331b0023069092e8e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:47:08.198969 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b5ade4-c54b-4d7d-ba1c-8446236214b0 service neutron] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:47:08.199252 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b5ade4-c54b-4d7d-ba1c-8446236214b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:47:08.199529 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b5ade4-c54b-4d7d-ba1c-8446236214b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:47:08.209883 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b5ade4-c54b-4d7d-ba1c-8446236214b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:47:08.211333 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b5ade4-c54b-4d7d-ba1c-8446236214b0 service neutron] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:47:08.221760 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1159/2344] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:47:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:47:12.842580 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e617837d-348e-4ee3-a928-640ae2fc45d2 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:47:12.843271 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e617837d-348e-4ee3-a928-640ae2fc45d2 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:47:12.843271 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e617837d-348e-4ee3-a928-640ae2fc45d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:47:12.874862 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-e617837d-348e-4ee3-a928-640ae2fc45d2 None None] MFA Rules not processed for user `ccc41bad96884148a0088cc8048a2316`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:47:12.876285 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-e617837d-348e-4ee3-a928-640ae2fc45d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:47:12.947707 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1186/2345] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:47:12 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:47:12.952998 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b5ade4-c54b-4d7d-ba1c-8446236214b0 service neutron] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:47:12.963322 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b5ade4-c54b-4d7d-ba1c-8446236214b0 service neutron] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:47:12.968916 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b4d181-20db-4b44-8a67-b81eff6b1c07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:47:12.969584 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b4d181-20db-4b44-8a67-b81eff6b1c07 admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:47:12.969860 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b4d181-20db-4b44-8a67-b81eff6b1c07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:47:12.970131 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b4d181-20db-4b44-8a67-b81eff6b1c07 admin admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:47:12.970991 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b4d181-20db-4b44-8a67-b81eff6b1c07 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:47:12.972884 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b4d181-20db-4b44-8a67-b81eff6b1c07 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:47:13.025473 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1160/2346] 10.222.0.89 () {60 vars in 1153 bytes} [Thu Jul 28 17:47:12 2022] POST /identity/v3/projects => generated 383 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:47:13.029455 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e617837d-348e-4ee3-a928-640ae2fc45d2 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:47:13.040972 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e617837d-348e-4ee3-a928-640ae2fc45d2 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:47:13.114845 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c396df5-c856-4112-a80c-ba5046732e3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:47:13.115474 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c396df5-c856-4112-a80c-ba5046732e3d admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:47:13.115773 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c396df5-c856-4112-a80c-ba5046732e3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:47:13.116037 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c396df5-c856-4112-a80c-ba5046732e3d admin admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:47:13.116707 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c396df5-c856-4112-a80c-ba5046732e3d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:47:13.118705 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c396df5-c856-4112-a80c-ba5046732e3d admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:47:13.157245 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1187/2347] 10.222.0.89 () {60 vars in 1144 bytes} [Thu Jul 28 17:47:13 2022] POST /identity/v3/users => generated 298 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:47:13.166877 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b4d181-20db-4b44-8a67-b81eff6b1c07 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:47:13.176433 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b4d181-20db-4b44-8a67-b81eff6b1c07 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:47:13.184363 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56192769-0acd-43bc-b217-1cea9fd371e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:47:13.185124 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56192769-0acd-43bc-b217-1cea9fd371e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:47:13.185491 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56192769-0acd-43bc-b217-1cea9fd371e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:47:13.185832 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56192769-0acd-43bc-b217-1cea9fd371e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:47:13.187252 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56192769-0acd-43bc-b217-1cea9fd371e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:47:13.189567 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56192769-0acd-43bc-b217-1cea9fd371e0 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:47:13.208366 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1161/2348] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:47:13 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:47:13.212410 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c396df5-c856-4112-a80c-ba5046732e3d admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:47:13.226429 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c396df5-c856-4112-a80c-ba5046732e3d admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:47:13.231699 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0dd8ce-80ac-4b34-b637-7132fbf83b35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:47:13.234477 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0dd8ce-80ac-4b34-b637-7132fbf83b35 admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:47:13.234775 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0dd8ce-80ac-4b34-b637-7132fbf83b35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:47:13.235055 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0dd8ce-80ac-4b34-b637-7132fbf83b35 admin admin] PATH_INFO: `/v3/projects/85eb8e598c144cf5a55a4ec5d99a027c/users/1206b722a36f4c66ad47f762a3e89420/roles/73e457231a824ea98c28d22610be640c` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:47:13.257890 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0dd8ce-80ac-4b34-b637-7132fbf83b35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=85eb8e598c144cf5a55a4ec5d99a027c, user_id=1206b722a36f4c66ad47f762a3e89420, role_id=73e457231a824ea98c28d22610be640c)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:47:13.260827 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0dd8ce-80ac-4b34-b637-7132fbf83b35 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:47:13.285774 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1188/2349] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:47:13 2022] PUT /identity/v3/projects/85eb8e598c144cf5a55a4ec5d99a027c/users/1206b722a36f4c66ad47f762a3e89420/roles/73e457231a824ea98c28d22610be640c => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:47:13.289681 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56192769-0acd-43bc-b217-1cea9fd371e0 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:47:13.303905 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56192769-0acd-43bc-b217-1cea9fd371e0 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:47:13.329009 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:47:13.372555 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e1970a-831d-45be-95dc-17475bd124ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:47:13.373409 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e1970a-831d-45be-95dc-17475bd124ca admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:47:13.373800 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e1970a-831d-45be-95dc-17475bd124ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:47:13.374333 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e1970a-831d-45be-95dc-17475bd124ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:47:13.375107 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e1970a-831d-45be-95dc-17475bd124ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:47:13.377423 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e1970a-831d-45be-95dc-17475bd124ca admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:47:13.390148 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1162/2350] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:47:13 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:47:13.393069 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0dd8ce-80ac-4b34-b637-7132fbf83b35 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:47:13.409413 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0dd8ce-80ac-4b34-b637-7132fbf83b35 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:47:13.413021 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e825c792-1fe9-4598-9091-0a6a16e3c410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:47:13.413815 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e825c792-1fe9-4598-9091-0a6a16e3c410 admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:47:13.414091 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e825c792-1fe9-4598-9091-0a6a16e3c410 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:47:13.414365 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e825c792-1fe9-4598-9091-0a6a16e3c410 admin admin] PATH_INFO: `/v3/projects/85eb8e598c144cf5a55a4ec5d99a027c/users/1206b722a36f4c66ad47f762a3e89420/roles/73e457231a824ea98c28d22610be640c` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:47:13.416407 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e825c792-1fe9-4598-9091-0a6a16e3c410 admin admin] RBAC: Authorizing `identity:create_grant(project_id=85eb8e598c144cf5a55a4ec5d99a027c, user_id=1206b722a36f4c66ad47f762a3e89420, role_id=73e457231a824ea98c28d22610be640c)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:47:13.419106 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e825c792-1fe9-4598-9091-0a6a16e3c410 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:47:13.432141 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1189/2351] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:47:13 2022] PUT /identity/v3/projects/85eb8e598c144cf5a55a4ec5d99a027c/users/1206b722a36f4c66ad47f762a3e89420/roles/73e457231a824ea98c28d22610be640c => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 17:47:18.931061 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61127ccb-1762-4dda-b4d6-d302daa9e604 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:47:18.931608 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61127ccb-1762-4dda-b4d6-d302daa9e604 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:47:18.931730 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61127ccb-1762-4dda-b4d6-d302daa9e604 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:47:18.960762 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-61127ccb-1762-4dda-b4d6-d302daa9e604 None None] MFA Rules not processed for user `1206b722a36f4c66ad47f762a3e89420`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:47:18.962146 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-61127ccb-1762-4dda-b4d6-d302daa9e604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:47:19.061982 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1163/2352] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:47:18 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:47:19.074360 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e825c792-1fe9-4598-9091-0a6a16e3c410 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:47:19.076519 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-e825c792-1fe9-4598-9091-0a6a16e3c410 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:47:19.084250 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:47:19.146885 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e825c792-1fe9-4598-9091-0a6a16e3c410 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:47:19.150510 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a084f7d-857c-4e8e-a1e1-e04f968f9d8e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1564db0771224a05b47d8e3cf778c147', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:47:19.151122 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a084f7d-857c-4e8e-a1e1-e04f968f9d8e service glance] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:47:19.151399 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a084f7d-857c-4e8e-a1e1-e04f968f9d8e service glance] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:47:19.151669 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a084f7d-857c-4e8e-a1e1-e04f968f9d8e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:47:19.218286 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a084f7d-857c-4e8e-a1e1-e04f968f9d8e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:47:19.219939 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a084f7d-857c-4e8e-a1e1-e04f968f9d8e service glance] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:47:19.229758 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1190/2353] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:47:19 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:47:19.401500 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e7a925-8431-4569-afe6-22e8749e6aae None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:47:19.401842 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e7a925-8431-4569-afe6-22e8749e6aae None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:47:19.402125 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e7a925-8431-4569-afe6-22e8749e6aae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:47:19.424218 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-e9e7a925-8431-4569-afe6-22e8749e6aae None None] MFA Rules not processed for user `c44387d13a694ee6bb47a13906003330`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:47:19.425740 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-e9e7a925-8431-4569-afe6-22e8749e6aae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:47:19.532526 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1164/2354] 10.222.0.89 () {58 vars in 1009 bytes} [Thu Jul 28 17:47:19 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:47:19.550175 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a084f7d-857c-4e8e-a1e1-e04f968f9d8e service glance] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:47:19.551355 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-4a084f7d-857c-4e8e-a1e1-e04f968f9d8e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:47:19.650724 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a084f7d-857c-4e8e-a1e1-e04f968f9d8e service glance] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:47:19.667587 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd903bfd-2c7e-4778-8819-258ecdc4cd44 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c59473b01bf4433ba3b111114fc048b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:47:19.668364 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd903bfd-2c7e-4778-8819-258ecdc4cd44 service swift] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:47:19.669402 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd903bfd-2c7e-4778-8819-258ecdc4cd44 service swift] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:47:19.670159 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd903bfd-2c7e-4778-8819-258ecdc4cd44 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:47:19.685278 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd903bfd-2c7e-4778-8819-258ecdc4cd44 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:47:19.687261 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd903bfd-2c7e-4778-8819-258ecdc4cd44 service swift] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:47:19.716614 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1191/2355] 10.222.0.89 () {60 vars in 1384 bytes} [Thu Jul 28 17:47:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 17:47:22.414537 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e7a925-8431-4569-afe6-22e8749e6aae None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:47:22.417006 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-e9e7a925-8431-4569-afe6-22e8749e6aae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:47:22.550000 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:47:22.576128 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e7a925-8431-4569-afe6-22e8749e6aae None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:47:22.581683 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a35b46-2486-49a1-9c31-81c6a7816ef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:47:22.582359 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a35b46-2486-49a1-9c31-81c6a7816ef8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:47:22.582639 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a35b46-2486-49a1-9c31-81c6a7816ef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:47:22.582918 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a35b46-2486-49a1-9c31-81c6a7816ef8 admin admin] PATH_INFO: `/v3/users/1ca38f2b90f043dbbb3fa27fa743c6bf` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:47:22.584133 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a35b46-2486-49a1-9c31-81c6a7816ef8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ca38f2b90f043dbbb3fa27fa743c6bf)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:47:22.586329 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a35b46-2486-49a1-9c31-81c6a7816ef8 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:47:22.662344 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-33a35b46-2486-49a1-9c31-81c6a7816ef8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ca38f2b90f043dbbb3fa27fa743c6bf'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:47:22.670333 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-33a35b46-2486-49a1-9c31-81c6a7816ef8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ca38f2b90f043dbbb3fa27fa743c6bf'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:47:22.703301 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-33a35b46-2486-49a1-9c31-81c6a7816ef8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ca38f2b90f043dbbb3fa27fa743c6bf'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:47:22.704119 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-33a35b46-2486-49a1-9c31-81c6a7816ef8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ca38f2b90f043dbbb3fa27fa743c6bf'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:47:22.704902 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-33a35b46-2486-49a1-9c31-81c6a7816ef8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ca38f2b90f043dbbb3fa27fa743c6bf'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:47:22.737926 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1165/2356] 10.222.0.89 () {58 vars in 1224 bytes} [Thu Jul 28 17:47:22 2022] DELETE /identity/v3/users/1ca38f2b90f043dbbb3fa27fa743c6bf => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:47:22.742076 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013c30b3-54c4-4da2-b695-e264fba41b7a None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:47:22.743128 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013c30b3-54c4-4da2-b695-e264fba41b7a None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:47:22.743633 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013c30b3-54c4-4da2-b695-e264fba41b7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:47:22.778491 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-013c30b3-54c4-4da2-b695-e264fba41b7a None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:47:22.782254 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-013c30b3-54c4-4da2-b695-e264fba41b7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:47:22.876801 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:47:22.910535 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1192/2357] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:47:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 17:47:22.928978 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a35b46-2486-49a1-9c31-81c6a7816ef8 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:47:22.930517 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-33a35b46-2486-49a1-9c31-81c6a7816ef8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:47:23.009758 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a35b46-2486-49a1-9c31-81c6a7816ef8 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:47:23.015105 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14529b84-9ccd-4bd9-9b8e-4857e3a7f3c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76817cd541294331b0023069092e8e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:47:23.016081 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14529b84-9ccd-4bd9-9b8e-4857e3a7f3c9 service neutron] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:47:23.016362 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14529b84-9ccd-4bd9-9b8e-4857e3a7f3c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:47:23.016635 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14529b84-9ccd-4bd9-9b8e-4857e3a7f3c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:47:23.027393 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14529b84-9ccd-4bd9-9b8e-4857e3a7f3c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:47:23.029457 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14529b84-9ccd-4bd9-9b8e-4857e3a7f3c9 service neutron] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:47:23.053344 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1166/2358] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:47:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:47:23.494720 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1025a346-0898-41ff-9585-4ac822af0e00 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:47:23.495023 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1025a346-0898-41ff-9585-4ac822af0e00 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:47:23.495270 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1025a346-0898-41ff-9585-4ac822af0e00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:47:23.526410 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-1025a346-0898-41ff-9585-4ac822af0e00 None None] MFA Rules not processed for user `c44387d13a694ee6bb47a13906003330`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:47:23.527703 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-1025a346-0898-41ff-9585-4ac822af0e00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:47:23.614895 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1193/2359] 10.222.0.89 () {58 vars in 1009 bytes} [Thu Jul 28 17:47:23 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:47:23.635276 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14529b84-9ccd-4bd9-9b8e-4857e3a7f3c9 service neutron] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:47:23.637518 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-14529b84-9ccd-4bd9-9b8e-4857e3a7f3c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:47:23.787968 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14529b84-9ccd-4bd9-9b8e-4857e3a7f3c9 service neutron] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:47:23.794079 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f98222f-ce35-46ff-9ed0-5a745c82b2eb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c59473b01bf4433ba3b111114fc048b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:47:23.794925 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f98222f-ce35-46ff-9ed0-5a745c82b2eb service swift] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:47:23.795309 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f98222f-ce35-46ff-9ed0-5a745c82b2eb service swift] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:47:23.796395 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f98222f-ce35-46ff-9ed0-5a745c82b2eb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:47:23.817929 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1025a346-0898-41ff-9585-4ac822af0e00 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:47:23.822403 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-1025a346-0898-41ff-9585-4ac822af0e00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:47:23.834601 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f98222f-ce35-46ff-9ed0-5a745c82b2eb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:47:23.836548 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f98222f-ce35-46ff-9ed0-5a745c82b2eb service swift] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:47:23.853716 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1025a346-0898-41ff-9585-4ac822af0e00 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:47:23.856275 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1167/2360] 10.222.0.89 () {60 vars in 1384 bytes} [Thu Jul 28 17:47:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 17:47:23.862513 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c172f7-9c93-4209-8022-e8c5094cd7ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:47:23.863390 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c172f7-9c93-4209-8022-e8c5094cd7ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:47:23.863876 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c172f7-9c93-4209-8022-e8c5094cd7ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:47:23.864309 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c172f7-9c93-4209-8022-e8c5094cd7ad admin admin] PATH_INFO: `/v3/projects/0cc8862d58e44881b784770c058f1ad2` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:47:23.865758 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c172f7-9c93-4209-8022-e8c5094cd7ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=0cc8862d58e44881b784770c058f1ad2)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:47:23.868387 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c172f7-9c93-4209-8022-e8c5094cd7ad admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:47:24.019044 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-c0c172f7-9c93-4209-8022-e8c5094cd7ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0cc8862d58e44881b784770c058f1ad2', 'request_id': 'req-c0c172f7-9c93-4209-8022-e8c5094cd7ad'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:47:24.033570 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-c0c172f7-9c93-4209-8022-e8c5094cd7ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0cc8862d58e44881b784770c058f1ad2', 'request_id': 'req-c0c172f7-9c93-4209-8022-e8c5094cd7ad'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:47:24.050035 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-c0c172f7-9c93-4209-8022-e8c5094cd7ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0cc8862d58e44881b784770c058f1ad2', 'request_id': 'req-c0c172f7-9c93-4209-8022-e8c5094cd7ad'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:47:24.097654 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-c0c172f7-9c93-4209-8022-e8c5094cd7ad admin admin] The token cache is being invalidate because project 0cc8862d58e44881b784770c058f1ad2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80324) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 17:47:24.098267 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-c0c172f7-9c93-4209-8022-e8c5094cd7ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:47:24.099270 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-c0c172f7-9c93-4209-8022-e8c5094cd7ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:47:24.102730 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1194/2361] 10.222.0.89 () {58 vars in 1233 bytes} [Thu Jul 28 17:47:23 2022] DELETE /identity/v3/projects/0cc8862d58e44881b784770c058f1ad2 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:47:24.107740 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cc9fa3-d0a9-4382-be72-df38d04449c4 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:47:24.108026 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cc9fa3-d0a9-4382-be72-df38d04449c4 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:47:24.108548 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cc9fa3-d0a9-4382-be72-df38d04449c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:47:24.120020 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:47:24.178640 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-e1cc9fa3-d0a9-4382-be72-df38d04449c4 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:47:24.179910 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-e1cc9fa3-d0a9-4382-be72-df38d04449c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:47:24.380376 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1168/2362] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:47:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:47:24.387832 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc90773c-c708-47cc-bfb5-8c0e940344df None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:47:24.388115 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc90773c-c708-47cc-bfb5-8c0e940344df None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:47:24.388647 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc90773c-c708-47cc-bfb5-8c0e940344df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:47:24.480078 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-cc90773c-c708-47cc-bfb5-8c0e940344df None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:47:24.482501 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-cc90773c-c708-47cc-bfb5-8c0e940344df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:47:24.503238 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1195/2363] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:47:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:47:24.506608 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cc9fa3-d0a9-4382-be72-df38d04449c4 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:47:24.521804 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cc9fa3-d0a9-4382-be72-df38d04449c4 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:47:24.541722 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9137b4-e8ae-496d-ad2e-840a7a2b0eed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:47:24.546593 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9137b4-e8ae-496d-ad2e-840a7a2b0eed admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:47:24.547045 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9137b4-e8ae-496d-ad2e-840a7a2b0eed admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:47:24.547331 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9137b4-e8ae-496d-ad2e-840a7a2b0eed admin admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:47:24.551736 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9137b4-e8ae-496d-ad2e-840a7a2b0eed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:47:24.552114 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9137b4-e8ae-496d-ad2e-840a7a2b0eed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:47:24.561449 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9137b4-e8ae-496d-ad2e-840a7a2b0eed admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:47:24.621874 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1169/2364] 10.222.0.89 () {58 vars in 1153 bytes} [Thu Jul 28 17:47:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:47:24.623508 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc90773c-c708-47cc-bfb5-8c0e940344df None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:47:24.639422 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc90773c-c708-47cc-bfb5-8c0e940344df None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:47:24.649939 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d83939-1bd4-41e7-acc1-5db18cb3f496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:47:24.650601 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d83939-1bd4-41e7-acc1-5db18cb3f496 admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:47:24.650868 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d83939-1bd4-41e7-acc1-5db18cb3f496 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:47:24.651129 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d83939-1bd4-41e7-acc1-5db18cb3f496 admin admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:47:24.656340 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d83939-1bd4-41e7-acc1-5db18cb3f496 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:47:24.660085 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d83939-1bd4-41e7-acc1-5db18cb3f496 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:47:24.739661 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1196/2365] 10.222.0.89 () {60 vars in 1153 bytes} [Thu Jul 28 17:47:24 2022] POST /identity/v3/projects => generated 371 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:47:24.741474 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9137b4-e8ae-496d-ad2e-840a7a2b0eed admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:47:24.756525 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9137b4-e8ae-496d-ad2e-840a7a2b0eed admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:47:24.860266 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb4db87-acb8-4731-b369-ab678e80c943 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:47:24.860928 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb4db87-acb8-4731-b369-ab678e80c943 admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:47:24.861320 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb4db87-acb8-4731-b369-ab678e80c943 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:47:24.861541 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb4db87-acb8-4731-b369-ab678e80c943 admin admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:47:24.862181 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb4db87-acb8-4731-b369-ab678e80c943 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:47:24.864136 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb4db87-acb8-4731-b369-ab678e80c943 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:47:24.900848 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1170/2366] 10.222.0.89 () {60 vars in 1144 bytes} [Thu Jul 28 17:47:24 2022] POST /identity/v3/users => generated 286 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:47:24.905247 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d83939-1bd4-41e7-acc1-5db18cb3f496 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:47:24.921518 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d83939-1bd4-41e7-acc1-5db18cb3f496 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:47:24.939211 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e82c53-f1e4-4c99-b03b-0e9ed97a13ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:47:24.939850 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e82c53-f1e4-4c99-b03b-0e9ed97a13ea admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:47:24.940132 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e82c53-f1e4-4c99-b03b-0e9ed97a13ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:47:24.940409 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e82c53-f1e4-4c99-b03b-0e9ed97a13ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:47:24.941724 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e82c53-f1e4-4c99-b03b-0e9ed97a13ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:47:24.943340 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e82c53-f1e4-4c99-b03b-0e9ed97a13ea admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:47:24.965343 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1197/2367] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:47:24 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 17:47:24.969485 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb4db87-acb8-4731-b369-ab678e80c943 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:47:24.989152 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb4db87-acb8-4731-b369-ab678e80c943 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:47:25.003650 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdd348d-c15e-413d-91eb-0af20e11995b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:47:25.004293 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdd348d-c15e-413d-91eb-0af20e11995b admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:47:25.004592 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdd348d-c15e-413d-91eb-0af20e11995b admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:47:25.004860 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdd348d-c15e-413d-91eb-0af20e11995b admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:47:25.005717 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdd348d-c15e-413d-91eb-0af20e11995b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:47:25.007476 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdd348d-c15e-413d-91eb-0af20e11995b admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:47:25.040075 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1171/2368] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:47:24 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:47:25.044131 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e82c53-f1e4-4c99-b03b-0e9ed97a13ea admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:47:25.068586 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e82c53-f1e4-4c99-b03b-0e9ed97a13ea admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:47:25.085628 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ab4228-0c42-4a3d-83f9-37c340c496de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:47:25.086347 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ab4228-0c42-4a3d-83f9-37c340c496de admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:47:25.087363 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ab4228-0c42-4a3d-83f9-37c340c496de admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:47:25.087655 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ab4228-0c42-4a3d-83f9-37c340c496de admin admin] PATH_INFO: `/v3/projects/0ef795b478944722886f1d084f49008d/users/1cd17607cef54135a91f39ccd74b6180/roles/11f8731756fe4caca13efa61b71b11c5` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:47:25.132274 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ab4228-0c42-4a3d-83f9-37c340c496de admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ef795b478944722886f1d084f49008d, user_id=1cd17607cef54135a91f39ccd74b6180, role_id=11f8731756fe4caca13efa61b71b11c5)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:47:25.135053 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ab4228-0c42-4a3d-83f9-37c340c496de admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:47:25.170489 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1198/2369] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:47:25 2022] PUT /identity/v3/projects/0ef795b478944722886f1d084f49008d/users/1cd17607cef54135a91f39ccd74b6180/roles/11f8731756fe4caca13efa61b71b11c5 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:47:25.176028 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb3a688-f000-48f2-a364-9877eab63bf4 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:47:25.176326 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb3a688-f000-48f2-a364-9877eab63bf4 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:47:25.176600 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb3a688-f000-48f2-a364-9877eab63bf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:47:25.204093 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-feb3a688-f000-48f2-a364-9877eab63bf4 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:47:25.205408 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-feb3a688-f000-48f2-a364-9877eab63bf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:47:25.395659 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1172/2370] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:47:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 17:47:25.406194 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ab4228-0c42-4a3d-83f9-37c340c496de admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:47:25.409199 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-65ab4228-0c42-4a3d-83f9-37c340c496de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:47:25.435087 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:47:25.560366 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ab4228-0c42-4a3d-83f9-37c340c496de admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:47:25.564790 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c644c8b-85ca-4e43-8d1f-5cf793877996 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76817cd541294331b0023069092e8e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:47:25.565684 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c644c8b-85ca-4e43-8d1f-5cf793877996 service neutron] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:47:25.566139 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c644c8b-85ca-4e43-8d1f-5cf793877996 service neutron] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:47:25.566568 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c644c8b-85ca-4e43-8d1f-5cf793877996 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:47:25.587770 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c644c8b-85ca-4e43-8d1f-5cf793877996 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:47:25.589784 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c644c8b-85ca-4e43-8d1f-5cf793877996 service neutron] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:47:25.616114 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1199/2371] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:47:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:47:34.977804 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10529d3b-f59f-4fba-9701-9cb16d0fe0db None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:47:34.978424 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10529d3b-f59f-4fba-9701-9cb16d0fe0db None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:47:34.978424 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10529d3b-f59f-4fba-9701-9cb16d0fe0db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:47:35.020999 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-10529d3b-f59f-4fba-9701-9cb16d0fe0db None None] MFA Rules not processed for user `ccc41bad96884148a0088cc8048a2316`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:47:35.022665 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-10529d3b-f59f-4fba-9701-9cb16d0fe0db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:47:35.027819 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:47:35.101081 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1173/2372] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:47:34 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:47:35.370961 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a2dae8-b1e1-49e7-baab-632eb420fc3f None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:47:35.371491 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a2dae8-b1e1-49e7-baab-632eb420fc3f None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:47:35.371674 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a2dae8-b1e1-49e7-baab-632eb420fc3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:47:35.427778 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-88a2dae8-b1e1-49e7-baab-632eb420fc3f None None] MFA Rules not processed for user `1cd17607cef54135a91f39ccd74b6180`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:47:35.429065 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-88a2dae8-b1e1-49e7-baab-632eb420fc3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:47:35.560706 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1200/2373] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:47:35 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:47:35.566020 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10529d3b-f59f-4fba-9701-9cb16d0fe0db None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:47:35.578407 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10529d3b-f59f-4fba-9701-9cb16d0fe0db None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:47:35.588483 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872c24e6-6296-478e-a859-d49f405e9679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:47:35.589897 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872c24e6-6296-478e-a859-d49f405e9679 admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:47:35.598269 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872c24e6-6296-478e-a859-d49f405e9679 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:47:35.598810 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872c24e6-6296-478e-a859-d49f405e9679 admin admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:47:35.622462 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872c24e6-6296-478e-a859-d49f405e9679 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:47:35.630408 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872c24e6-6296-478e-a859-d49f405e9679 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:47:35.676869 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1174/2374] 10.222.0.89 () {60 vars in 1153 bytes} [Thu Jul 28 17:47:35 2022] POST /identity/v3/projects => generated 371 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:47:35.677733 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a2dae8-b1e1-49e7-baab-632eb420fc3f None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:47:35.711664 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a2dae8-b1e1-49e7-baab-632eb420fc3f None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:47:35.860743 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fad86b-6969-44b9-b806-04d6ad5241df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:47:35.861505 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fad86b-6969-44b9-b806-04d6ad5241df admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:47:35.861910 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fad86b-6969-44b9-b806-04d6ad5241df admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:47:35.862251 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fad86b-6969-44b9-b806-04d6ad5241df admin admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:47:35.863329 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fad86b-6969-44b9-b806-04d6ad5241df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:47:35.865317 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fad86b-6969-44b9-b806-04d6ad5241df admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:47:35.914068 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872c24e6-6296-478e-a859-d49f405e9679 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:47:35.924769 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1201/2375] 10.222.0.89 () {60 vars in 1144 bytes} [Thu Jul 28 17:47:35 2022] POST /identity/v3/users => generated 292 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:47:35.929306 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872c24e6-6296-478e-a859-d49f405e9679 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:47:35.941586 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af75091-c003-4140-a1e1-5d0a8ce61c32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:47:35.942293 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af75091-c003-4140-a1e1-5d0a8ce61c32 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:47:35.942483 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af75091-c003-4140-a1e1-5d0a8ce61c32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:47:35.942754 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af75091-c003-4140-a1e1-5d0a8ce61c32 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:47:35.943665 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af75091-c003-4140-a1e1-5d0a8ce61c32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:47:35.949387 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af75091-c003-4140-a1e1-5d0a8ce61c32 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:47:35.966206 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1175/2376] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:47:35 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:47:35.971637 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fad86b-6969-44b9-b806-04d6ad5241df admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:47:35.987010 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fad86b-6969-44b9-b806-04d6ad5241df admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:47:36.005995 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403fc584-2e61-411a-ac37-5e56a7c43ddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:47:36.006942 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403fc584-2e61-411a-ac37-5e56a7c43ddb admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:47:36.007992 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403fc584-2e61-411a-ac37-5e56a7c43ddb admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:47:36.008476 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403fc584-2e61-411a-ac37-5e56a7c43ddb admin admin] PATH_INFO: `/v3/projects/54265825d5534d4b9f49e5633615db1d/users/7f925222c6f04622b5ee22c10f0ba48b/roles/73e457231a824ea98c28d22610be640c` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:47:36.071181 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403fc584-2e61-411a-ac37-5e56a7c43ddb admin admin] RBAC: Authorizing `identity:create_grant(project_id=54265825d5534d4b9f49e5633615db1d, user_id=7f925222c6f04622b5ee22c10f0ba48b, role_id=73e457231a824ea98c28d22610be640c)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:47:36.074412 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403fc584-2e61-411a-ac37-5e56a7c43ddb admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:47:36.116033 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1202/2377] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:47:35 2022] PUT /identity/v3/projects/54265825d5534d4b9f49e5633615db1d/users/7f925222c6f04622b5ee22c10f0ba48b/roles/73e457231a824ea98c28d22610be640c => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:47:36.130583 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af75091-c003-4140-a1e1-5d0a8ce61c32 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:47:36.157760 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af75091-c003-4140-a1e1-5d0a8ce61c32 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:47:36.330556 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d575b8-c618-4d1f-8889-2a6d5ff497f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:47:36.331192 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d575b8-c618-4d1f-8889-2a6d5ff497f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:47:36.331477 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d575b8-c618-4d1f-8889-2a6d5ff497f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:47:36.331899 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d575b8-c618-4d1f-8889-2a6d5ff497f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:47:36.332619 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d575b8-c618-4d1f-8889-2a6d5ff497f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:47:36.340112 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d575b8-c618-4d1f-8889-2a6d5ff497f9 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:47:36.363043 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1176/2378] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:47:36 2022] GET /identity/v3/roles => generated 1420 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:47:36.363553 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403fc584-2e61-411a-ac37-5e56a7c43ddb admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:47:36.389250 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403fc584-2e61-411a-ac37-5e56a7c43ddb admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:47:36.407577 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324ffa04-7da4-45b8-964a-dce8fc836ba9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:47:36.408561 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324ffa04-7da4-45b8-964a-dce8fc836ba9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:47:36.408872 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324ffa04-7da4-45b8-964a-dce8fc836ba9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:47:36.409116 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324ffa04-7da4-45b8-964a-dce8fc836ba9 admin admin] PATH_INFO: `/v3/projects/54265825d5534d4b9f49e5633615db1d/users/7f925222c6f04622b5ee22c10f0ba48b/roles/73e457231a824ea98c28d22610be640c` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:47:36.414324 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324ffa04-7da4-45b8-964a-dce8fc836ba9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=54265825d5534d4b9f49e5633615db1d, user_id=7f925222c6f04622b5ee22c10f0ba48b, role_id=73e457231a824ea98c28d22610be640c)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:47:36.417361 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324ffa04-7da4-45b8-964a-dce8fc836ba9 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:47:36.424763 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:47:36.446029 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1203/2379] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:47:36 2022] PUT /identity/v3/projects/54265825d5534d4b9f49e5633615db1d/users/7f925222c6f04622b5ee22c10f0ba48b/roles/73e457231a824ea98c28d22610be640c => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 17:47:36.980527 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d575b8-c618-4d1f-8889-2a6d5ff497f9 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:47:36.982820 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-08d575b8-c618-4d1f-8889-2a6d5ff497f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:47:37.080096 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d575b8-c618-4d1f-8889-2a6d5ff497f9 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:47:37.110914 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:47:37.232340 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ac4f25-795c-40bc-8baf-5bd74f440cfa service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4bd290018394b0d955cd376ce35b77f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:47:37.234554 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ac4f25-795c-40bc-8baf-5bd74f440cfa service placement] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:47:37.235029 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ac4f25-795c-40bc-8baf-5bd74f440cfa service placement] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:47:37.235481 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ac4f25-795c-40bc-8baf-5bd74f440cfa service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:47:37.238304 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-a4ac4f25-795c-40bc-8baf-5bd74f440cfa service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:47:37.267421 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ac4f25-795c-40bc-8baf-5bd74f440cfa service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:47:37.269684 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ac4f25-795c-40bc-8baf-5bd74f440cfa service placement] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:47:37.300078 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1177/2380] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:47:36 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:47:39.117012 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af1d1e7-3662-465f-867a-7f8939d6519d None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:47:39.118307 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af1d1e7-3662-465f-867a-7f8939d6519d None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:47:39.118847 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af1d1e7-3662-465f-867a-7f8939d6519d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:47:39.189150 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-1af1d1e7-3662-465f-867a-7f8939d6519d None None] MFA Rules not processed for user `c44387d13a694ee6bb47a13906003330`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:47:39.196656 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-1af1d1e7-3662-465f-867a-7f8939d6519d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:47:39.371998 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1204/2381] 10.222.0.89 () {58 vars in 1009 bytes} [Thu Jul 28 17:47:39 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 17:47:39.455389 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ac4f25-795c-40bc-8baf-5bd74f440cfa service placement] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:47:39.457272 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-a4ac4f25-795c-40bc-8baf-5bd74f440cfa service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:47:39.744098 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ac4f25-795c-40bc-8baf-5bd74f440cfa service placement] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:47:39.752950 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7666aa-4014-45fc-94de-540c9972b0b3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c59473b01bf4433ba3b111114fc048b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:47:39.754687 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7666aa-4014-45fc-94de-540c9972b0b3 service swift] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:47:39.755141 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7666aa-4014-45fc-94de-540c9972b0b3 service swift] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:47:39.755576 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7666aa-4014-45fc-94de-540c9972b0b3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:47:39.782508 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7666aa-4014-45fc-94de-540c9972b0b3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:47:39.786172 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7666aa-4014-45fc-94de-540c9972b0b3 service swift] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:47:39.856125 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1178/2382] 10.222.0.89 () {60 vars in 1384 bytes} [Thu Jul 28 17:47:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 17:47:45.148404 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211b7dc9-13e6-4102-85cc-c808e42e96e9 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:47:45.148869 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211b7dc9-13e6-4102-85cc-c808e42e96e9 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:47:45.148869 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211b7dc9-13e6-4102-85cc-c808e42e96e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:47:45.197966 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-211b7dc9-13e6-4102-85cc-c808e42e96e9 None None] MFA Rules not processed for user `7f925222c6f04622b5ee22c10f0ba48b`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:47:45.199894 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-211b7dc9-13e6-4102-85cc-c808e42e96e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:47:45.357510 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1205/2383] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:47:45 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:47:45.372784 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7666aa-4014-45fc-94de-540c9972b0b3 service swift] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:47:45.375156 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-7a7666aa-4014-45fc-94de-540c9972b0b3 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:47:45.479364 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7666aa-4014-45fc-94de-540c9972b0b3 service swift] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:47:45.489317 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64ad7c0-be95-4f5e-8175-1cb9910052cc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd10b49b29df94404a71f43273839c252', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:47:45.490209 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64ad7c0-be95-4f5e-8175-1cb9910052cc service nova] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:47:45.490556 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64ad7c0-be95-4f5e-8175-1cb9910052cc service nova] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:47:45.490897 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64ad7c0-be95-4f5e-8175-1cb9910052cc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:47:45.561326 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:47:45.630158 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64ad7c0-be95-4f5e-8175-1cb9910052cc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:47:45.631692 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64ad7c0-be95-4f5e-8175-1cb9910052cc service nova] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:47:45.642916 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1179/2384] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:47:45 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:47:47.400702 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005b31d4-21d5-4804-b90f-7031a6de0262 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:47:47.401673 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005b31d4-21d5-4804-b90f-7031a6de0262 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:47:47.402153 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005b31d4-21d5-4804-b90f-7031a6de0262 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:47:47.443639 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-005b31d4-21d5-4804-b90f-7031a6de0262 None None] MFA Rules not processed for user `1cd17607cef54135a91f39ccd74b6180`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:47:47.446568 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-005b31d4-21d5-4804-b90f-7031a6de0262 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:47:47.469404 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1206/2385] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:47:47 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:47:55.637858 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=11 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:47:57.432982 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=64 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:47:59.379839 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64ad7c0-be95-4f5e-8175-1cb9910052cc service nova] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:47:59.381475 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-f64ad7c0-be95-4f5e-8175-1cb9910052cc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:47:59.456716 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64ad7c0-be95-4f5e-8175-1cb9910052cc service nova] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:47:59.462525 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f1eba4-938d-4fbf-9bd1-cda15a36ed9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:47:59.463210 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f1eba4-938d-4fbf-9bd1-cda15a36ed9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:47:59.463543 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f1eba4-938d-4fbf-9bd1-cda15a36ed9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:47:59.463806 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f1eba4-938d-4fbf-9bd1-cda15a36ed9c admin admin] PATH_INFO: `/v3/users/88d61d95ab7a4e5c9413ee94fc936d34` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:47:59.464979 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f1eba4-938d-4fbf-9bd1-cda15a36ed9c admin admin] RBAC: Authorizing `identity:delete_user(user_id=88d61d95ab7a4e5c9413ee94fc936d34)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:47:59.466985 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f1eba4-938d-4fbf-9bd1-cda15a36ed9c admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:47:59.522436 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-81f1eba4-938d-4fbf-9bd1-cda15a36ed9c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88d61d95ab7a4e5c9413ee94fc936d34'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:47:59.529274 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-81f1eba4-938d-4fbf-9bd1-cda15a36ed9c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88d61d95ab7a4e5c9413ee94fc936d34'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:47:59.538199 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-81f1eba4-938d-4fbf-9bd1-cda15a36ed9c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88d61d95ab7a4e5c9413ee94fc936d34'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:47:59.538874 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-81f1eba4-938d-4fbf-9bd1-cda15a36ed9c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88d61d95ab7a4e5c9413ee94fc936d34'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:47:59.539493 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-81f1eba4-938d-4fbf-9bd1-cda15a36ed9c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88d61d95ab7a4e5c9413ee94fc936d34'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:47:59.558983 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1180/2386] 10.222.0.89 () {58 vars in 1224 bytes} [Thu Jul 28 17:47:59 2022] DELETE /identity/v3/users/88d61d95ab7a4e5c9413ee94fc936d34 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:47:59.564324 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dab629-b898-4ebb-8e19-fa9695dc0d59 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:47:59.564844 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dab629-b898-4ebb-8e19-fa9695dc0d59 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:47:59.564844 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dab629-b898-4ebb-8e19-fa9695dc0d59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:47:59.584682 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-26dab629-b898-4ebb-8e19-fa9695dc0d59 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:47:59.585920 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-26dab629-b898-4ebb-8e19-fa9695dc0d59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:47:59.663982 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1207/2387] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:47:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:47:59.675434 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f1eba4-938d-4fbf-9bd1-cda15a36ed9c admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:47:59.677240 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-81f1eba4-938d-4fbf-9bd1-cda15a36ed9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:47:59.746032 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f1eba4-938d-4fbf-9bd1-cda15a36ed9c admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:47:59.750035 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7c4136-9306-4254-bf0d-3a74c53bbd51 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76817cd541294331b0023069092e8e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:47:59.751804 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7c4136-9306-4254-bf0d-3a74c53bbd51 service neutron] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:47:59.752090 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7c4136-9306-4254-bf0d-3a74c53bbd51 service neutron] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:47:59.752363 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7c4136-9306-4254-bf0d-3a74c53bbd51 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:47:59.763308 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7c4136-9306-4254-bf0d-3a74c53bbd51 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:47:59.767070 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7c4136-9306-4254-bf0d-3a74c53bbd51 service neutron] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:47:59.777545 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1181/2388] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:47:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 17:48:00.222357 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dab629-b898-4ebb-8e19-fa9695dc0d59 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:48:00.224053 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-26dab629-b898-4ebb-8e19-fa9695dc0d59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:48:00.241756 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dab629-b898-4ebb-8e19-fa9695dc0d59 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:48:00.251601 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2c6475-c9d8-4e59-b13b-3b9ffb90a304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:48:00.252576 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2c6475-c9d8-4e59-b13b-3b9ffb90a304 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:48:00.253262 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2c6475-c9d8-4e59-b13b-3b9ffb90a304 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:48:00.265339 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2c6475-c9d8-4e59-b13b-3b9ffb90a304 admin admin] PATH_INFO: `/v3/projects/b4e1c233487d48c69cdbbd059a0c3108` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:48:00.270945 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2c6475-c9d8-4e59-b13b-3b9ffb90a304 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4e1c233487d48c69cdbbd059a0c3108)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:48:00.273485 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2c6475-c9d8-4e59-b13b-3b9ffb90a304 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:48:00.325877 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-7c2c6475-c9d8-4e59-b13b-3b9ffb90a304 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4e1c233487d48c69cdbbd059a0c3108', 'request_id': 'req-7c2c6475-c9d8-4e59-b13b-3b9ffb90a304'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:48:00.335635 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-7c2c6475-c9d8-4e59-b13b-3b9ffb90a304 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4e1c233487d48c69cdbbd059a0c3108', 'request_id': 'req-7c2c6475-c9d8-4e59-b13b-3b9ffb90a304'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:48:00.342756 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-7c2c6475-c9d8-4e59-b13b-3b9ffb90a304 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4e1c233487d48c69cdbbd059a0c3108', 'request_id': 'req-7c2c6475-c9d8-4e59-b13b-3b9ffb90a304'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:48:00.361774 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-7c2c6475-c9d8-4e59-b13b-3b9ffb90a304 admin admin] The token cache is being invalidate because project b4e1c233487d48c69cdbbd059a0c3108 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80324) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 17:48:00.362221 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-7c2c6475-c9d8-4e59-b13b-3b9ffb90a304 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:48:00.362889 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-7c2c6475-c9d8-4e59-b13b-3b9ffb90a304 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:48:00.364420 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1208/2389] 10.222.0.89 () {58 vars in 1233 bytes} [Thu Jul 28 17:48:00 2022] DELETE /identity/v3/projects/b4e1c233487d48c69cdbbd059a0c3108 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:48:00.369618 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce779e9-fdbb-413d-9cd4-b4b02f9b7c9d None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:48:00.369900 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce779e9-fdbb-413d-9cd4-b4b02f9b7c9d None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:48:00.370228 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce779e9-fdbb-413d-9cd4-b4b02f9b7c9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:48:00.393719 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-1ce779e9-fdbb-413d-9cd4-b4b02f9b7c9d None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:48:00.395094 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-1ce779e9-fdbb-413d-9cd4-b4b02f9b7c9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:48:00.418812 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:48:00.494140 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1182/2390] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:48:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 17:48:00.502802 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae45773-af01-4eef-94ce-2d6a4da0c50d None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:48:00.503087 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae45773-af01-4eef-94ce-2d6a4da0c50d None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:48:00.503369 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae45773-af01-4eef-94ce-2d6a4da0c50d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:48:00.542307 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-2ae45773-af01-4eef-94ce-2d6a4da0c50d None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:48:00.543594 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-2ae45773-af01-4eef-94ce-2d6a4da0c50d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:48:00.577693 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1209/2391] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:48:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:48:00.579932 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce779e9-fdbb-413d-9cd4-b4b02f9b7c9d None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:48:00.595822 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce779e9-fdbb-413d-9cd4-b4b02f9b7c9d None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:48:00.601893 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0fd300-0341-49da-a053-91f1bcdb7170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:48:00.602544 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0fd300-0341-49da-a053-91f1bcdb7170 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:48:00.602806 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0fd300-0341-49da-a053-91f1bcdb7170 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:48:00.603081 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0fd300-0341-49da-a053-91f1bcdb7170 admin admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:48:00.603924 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0fd300-0341-49da-a053-91f1bcdb7170 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:48:00.604231 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0fd300-0341-49da-a053-91f1bcdb7170 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:48:00.606206 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0fd300-0341-49da-a053-91f1bcdb7170 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:48:00.622390 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1183/2392] 10.222.0.89 () {58 vars in 1153 bytes} [Thu Jul 28 17:48:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:48:00.627290 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae45773-af01-4eef-94ce-2d6a4da0c50d None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:48:00.637328 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae45773-af01-4eef-94ce-2d6a4da0c50d None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:48:00.649204 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708c2c8b-5a23-48ad-b3f4-b7da7b109118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:48:00.649779 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708c2c8b-5a23-48ad-b3f4-b7da7b109118 admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:48:00.650055 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708c2c8b-5a23-48ad-b3f4-b7da7b109118 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:48:00.650321 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708c2c8b-5a23-48ad-b3f4-b7da7b109118 admin admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:48:00.651153 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708c2c8b-5a23-48ad-b3f4-b7da7b109118 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:48:00.653184 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708c2c8b-5a23-48ad-b3f4-b7da7b109118 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:48:00.704651 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1210/2393] 10.222.0.89 () {60 vars in 1153 bytes} [Thu Jul 28 17:48:00 2022] POST /identity/v3/projects => generated 371 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:48:00.709208 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0fd300-0341-49da-a053-91f1bcdb7170 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:48:00.723550 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54d2600-536a-4466-a354-c393f1d5b082 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:48:00.723871 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54d2600-536a-4466-a354-c393f1d5b082 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:48:00.724108 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54d2600-536a-4466-a354-c393f1d5b082 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:48:00.726766 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1211/2394] 10.222.0.89 () {54 vars in 921 bytes} [Thu Jul 28 17:48:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 17:48:00.731623 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0fd300-0341-49da-a053-91f1bcdb7170 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:48:00.737891 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3fbc91-75ea-4a2c-b9cd-26535a01661a None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:48:00.738430 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3fbc91-75ea-4a2c-b9cd-26535a01661a None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:48:00.739164 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3fbc91-75ea-4a2c-b9cd-26535a01661a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:48:00.763971 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-6b3fbc91-75ea-4a2c-b9cd-26535a01661a None None] MFA Rules not processed for user `d10b49b29df94404a71f43273839c252`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:48:00.765786 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-6b3fbc91-75ea-4a2c-b9cd-26535a01661a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:48:00.800019 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4506156e-a662-47f7-a65b-3ace26563e77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:48:00.800731 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4506156e-a662-47f7-a65b-3ace26563e77 admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:48:00.801080 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4506156e-a662-47f7-a65b-3ace26563e77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:48:00.801478 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4506156e-a662-47f7-a65b-3ace26563e77 admin admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:48:00.802390 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4506156e-a662-47f7-a65b-3ace26563e77 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:48:00.804555 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4506156e-a662-47f7-a65b-3ace26563e77 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:48:00.832578 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1184/2395] 10.222.0.89 () {60 vars in 1144 bytes} [Thu Jul 28 17:48:00 2022] POST /identity/v3/users => generated 286 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 17:48:00.835281 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:48:00.840202 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4506156e-a662-47f7-a65b-3ace26563e77 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:48:00.870944 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1212/2396] 10.222.0.89 () {58 vars in 1018 bytes} [Thu Jul 28 17:48:00 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:48:00.874185 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4506156e-a662-47f7-a65b-3ace26563e77 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:48:00.883397 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157122c9-afb8-4fcf-bc50-1c923b112832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:48:00.884200 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157122c9-afb8-4fcf-bc50-1c923b112832 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:48:00.884706 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157122c9-afb8-4fcf-bc50-1c923b112832 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:48:00.885911 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157122c9-afb8-4fcf-bc50-1c923b112832 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:48:00.886987 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157122c9-afb8-4fcf-bc50-1c923b112832 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:48:00.889708 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157122c9-afb8-4fcf-bc50-1c923b112832 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:48:00.891552 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3fbc91-75ea-4a2c-b9cd-26535a01661a None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:48:00.892974 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-6b3fbc91-75ea-4a2c-b9cd-26535a01661a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:48:00.918090 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1185/2397] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:48:00 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 17:48:00.919675 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157122c9-afb8-4fcf-bc50-1c923b112832 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:48:00.944942 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157122c9-afb8-4fcf-bc50-1c923b112832 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:48:00.950359 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ea5948-817a-413c-bb81-e4a09c4de008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:48:00.950991 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ea5948-817a-413c-bb81-e4a09c4de008 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:48:00.951269 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ea5948-817a-413c-bb81-e4a09c4de008 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:48:00.951545 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ea5948-817a-413c-bb81-e4a09c4de008 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:48:00.952230 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ea5948-817a-413c-bb81-e4a09c4de008 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:48:00.954407 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ea5948-817a-413c-bb81-e4a09c4de008 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:48:00.964549 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1186/2398] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:48:00 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:48:00.968589 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ea5948-817a-413c-bb81-e4a09c4de008 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:48:00.977838 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3fbc91-75ea-4a2c-b9cd-26535a01661a None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:48:00.979429 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ea5948-817a-413c-bb81-e4a09c4de008 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:48:00.981572 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d80b94-c98d-4c56-9a89-92c9d6f170f5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1564db0771224a05b47d8e3cf778c147', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:48:00.982199 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d80b94-c98d-4c56-9a89-92c9d6f170f5 service glance] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:48:00.982468 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d80b94-c98d-4c56-9a89-92c9d6f170f5 service glance] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:48:00.982741 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d80b94-c98d-4c56-9a89-92c9d6f170f5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:48:00.986807 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4ad0e2-8ed4-4c84-897e-04d9568105f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:48:00.987696 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4ad0e2-8ed4-4c84-897e-04d9568105f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:48:00.987981 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4ad0e2-8ed4-4c84-897e-04d9568105f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:48:00.988353 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4ad0e2-8ed4-4c84-897e-04d9568105f2 admin admin] PATH_INFO: `/v3/projects/dcc5c66384d546d298795f25eabcc5c9/users/87ed2df69d9d4dc58aead0b1e0fbceea/roles/11f8731756fe4caca13efa61b71b11c5` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:48:00.996346 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d80b94-c98d-4c56-9a89-92c9d6f170f5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:48:00.997822 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d80b94-c98d-4c56-9a89-92c9d6f170f5 service glance] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:48:01.005558 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4ad0e2-8ed4-4c84-897e-04d9568105f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dcc5c66384d546d298795f25eabcc5c9, user_id=87ed2df69d9d4dc58aead0b1e0fbceea, role_id=11f8731756fe4caca13efa61b71b11c5)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:48:01.008222 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4ad0e2-8ed4-4c84-897e-04d9568105f2 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:48:01.009797 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1213/2399] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:48:00 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:48:01.021924 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1187/2400] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:48:00 2022] PUT /identity/v3/projects/dcc5c66384d546d298795f25eabcc5c9/users/87ed2df69d9d4dc58aead0b1e0fbceea/roles/11f8731756fe4caca13efa61b71b11c5 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 17:48:01.032534 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93492e5-655e-40f2-be57-50e285d26f24 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:48:01.032856 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93492e5-655e-40f2-be57-50e285d26f24 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:48:01.033198 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93492e5-655e-40f2-be57-50e285d26f24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:48:01.063406 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-a93492e5-655e-40f2-be57-50e285d26f24 None None] MFA Rules not processed for user `87ed2df69d9d4dc58aead0b1e0fbceea`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:48:01.064889 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-a93492e5-655e-40f2-be57-50e285d26f24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:48:01.086317 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ed8212-f72e-4f6f-917d-545ce6124928 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:48:01.087056 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ed8212-f72e-4f6f-917d-545ce6124928 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:48:01.087375 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ed8212-f72e-4f6f-917d-545ce6124928 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:48:01.088780 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1188/2401] 10.222.0.89 () {54 vars in 921 bytes} [Thu Jul 28 17:48:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 17:48:01.093486 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657ba344-99e0-4c33-8735-152cc11ff8a4 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:48:01.093768 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657ba344-99e0-4c33-8735-152cc11ff8a4 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:48:01.094561 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657ba344-99e0-4c33-8735-152cc11ff8a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:48:01.116883 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-657ba344-99e0-4c33-8735-152cc11ff8a4 None None] MFA Rules not processed for user `76817cd541294331b0023069092e8e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:48:01.118666 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-657ba344-99e0-4c33-8735-152cc11ff8a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:48:01.134281 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1214/2402] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:48:01 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:48:01.148991 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93492e5-655e-40f2-be57-50e285d26f24 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:48:01.182157 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1189/2403] 10.222.0.89 () {58 vars in 1018 bytes} [Thu Jul 28 17:48:01 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:48:01.195045 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657ba344-99e0-4c33-8735-152cc11ff8a4 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:48:01.196542 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-657ba344-99e0-4c33-8735-152cc11ff8a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:48:01.202392 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:48:01.209224 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657ba344-99e0-4c33-8735-152cc11ff8a4 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:48:01.213749 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baedfbb-9f21-4e74-acc7-3b5e16ee0746 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76817cd541294331b0023069092e8e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:48:01.214361 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baedfbb-9f21-4e74-acc7-3b5e16ee0746 service neutron] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:48:01.214638 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baedfbb-9f21-4e74-acc7-3b5e16ee0746 service neutron] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:48:01.214911 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baedfbb-9f21-4e74-acc7-3b5e16ee0746 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:48:01.216039 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93492e5-655e-40f2-be57-50e285d26f24 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:48:01.220612 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fed5c8-f521-46d4-8100-a4efdb7939d7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1564db0771224a05b47d8e3cf778c147', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:48:01.221255 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fed5c8-f521-46d4-8100-a4efdb7939d7 service glance] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:48:01.221540 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fed5c8-f521-46d4-8100-a4efdb7939d7 service glance] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:48:01.221838 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fed5c8-f521-46d4-8100-a4efdb7939d7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:48:01.223459 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9baedfbb-9f21-4e74-acc7-3b5e16ee0746 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:48:01.225291 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9baedfbb-9f21-4e74-acc7-3b5e16ee0746 service neutron] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:48:01.230526 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fed5c8-f521-46d4-8100-a4efdb7939d7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:48:01.232205 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fed5c8-f521-46d4-8100-a4efdb7939d7 service glance] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:48:01.233425 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1190/2404] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:48:01 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:48:01.245008 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1215/2405] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:48:01 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 17:48:01.354076 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dbc688-e04f-48c3-abe1-b4f8900d15e7 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:48:01.354366 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dbc688-e04f-48c3-abe1-b4f8900d15e7 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:48:01.354707 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dbc688-e04f-48c3-abe1-b4f8900d15e7 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:48:01.356060 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1191/2406] 10.222.0.89 () {54 vars in 921 bytes} [Thu Jul 28 17:48:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:48:01.363182 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3d38e5-acda-4b4c-bb0c-aadbf74b9828 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:48:01.363499 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3d38e5-acda-4b4c-bb0c-aadbf74b9828 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:48:01.363783 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3d38e5-acda-4b4c-bb0c-aadbf74b9828 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:48:01.384071 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-7d3d38e5-acda-4b4c-bb0c-aadbf74b9828 None None] MFA Rules not processed for user `d10b49b29df94404a71f43273839c252`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:48:01.385326 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-7d3d38e5-acda-4b4c-bb0c-aadbf74b9828 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:48:01.404462 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:48:01.460255 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1216/2407] 10.222.0.89 () {58 vars in 1018 bytes} [Thu Jul 28 17:48:01 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:48:01.472363 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dbc688-e04f-48c3-abe1-b4f8900d15e7 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:48:01.489102 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dbc688-e04f-48c3-abe1-b4f8900d15e7 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:48:01.492924 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364df27c-be11-4e5d-aa20-d3d5d8a8ad30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76817cd541294331b0023069092e8e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:48:01.493597 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364df27c-be11-4e5d-aa20-d3d5d8a8ad30 service neutron] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:48:01.493875 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364df27c-be11-4e5d-aa20-d3d5d8a8ad30 service neutron] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:48:01.494191 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364df27c-be11-4e5d-aa20-d3d5d8a8ad30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:48:01.510483 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364df27c-be11-4e5d-aa20-d3d5d8a8ad30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:48:01.512204 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364df27c-be11-4e5d-aa20-d3d5d8a8ad30 service neutron] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:48:01.525029 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1192/2408] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:48:01 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 17:48:09.539696 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc5bd44-1a53-49e9-b423-c18ac568032a None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:48:09.540156 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc5bd44-1a53-49e9-b423-c18ac568032a None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:48:09.540247 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc5bd44-1a53-49e9-b423-c18ac568032a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:48:09.560310 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-5fc5bd44-1a53-49e9-b423-c18ac568032a None None] MFA Rules not processed for user `c44387d13a694ee6bb47a13906003330`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:48:09.561559 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-5fc5bd44-1a53-49e9-b423-c18ac568032a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:48:09.617817 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1217/2409] 10.222.0.89 () {58 vars in 1009 bytes} [Thu Jul 28 17:48:09 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:48:09.640504 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364df27c-be11-4e5d-aa20-d3d5d8a8ad30 service neutron] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:48:09.641921 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-364df27c-be11-4e5d-aa20-d3d5d8a8ad30 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:48:09.714286 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364df27c-be11-4e5d-aa20-d3d5d8a8ad30 service neutron] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:48:09.719282 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951e0648-ada8-440f-98a2-585d40a3928a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c59473b01bf4433ba3b111114fc048b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:48:09.719890 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951e0648-ada8-440f-98a2-585d40a3928a service swift] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:48:09.720688 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951e0648-ada8-440f-98a2-585d40a3928a service swift] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:48:09.721287 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951e0648-ada8-440f-98a2-585d40a3928a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:48:09.728928 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951e0648-ada8-440f-98a2-585d40a3928a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:48:09.730579 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951e0648-ada8-440f-98a2-585d40a3928a service swift] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:48:09.739487 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1193/2410] 10.222.0.89 () {60 vars in 1384 bytes} [Thu Jul 28 17:48:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 17:48:19.609726 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=46 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:48:19.735401 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=43 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:48:23.122093 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc5bd44-1a53-49e9-b423-c18ac568032a None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:48:23.131868 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc5bd44-1a53-49e9-b423-c18ac568032a None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:48:23.135571 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f43587-eb45-48b0-af18-75d34bc02c2a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76817cd541294331b0023069092e8e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:48:23.136254 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f43587-eb45-48b0-af18-75d34bc02c2a service neutron] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:48:23.136601 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f43587-eb45-48b0-af18-75d34bc02c2a service neutron] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:48:23.136938 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f43587-eb45-48b0-af18-75d34bc02c2a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:48:23.138892 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-c9f43587-eb45-48b0-af18-75d34bc02c2a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:48:23.206721 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f43587-eb45-48b0-af18-75d34bc02c2a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:48:23.208587 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f43587-eb45-48b0-af18-75d34bc02c2a service neutron] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:48:23.218751 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1218/2411] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:48:23 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:48:33.214104 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=30 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:48:50.405980 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951e0648-ada8-440f-98a2-585d40a3928a service swift] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:48:50.475609 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951e0648-ada8-440f-98a2-585d40a3928a service swift] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:48:50.480568 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866098ae-7b00-4716-8b0b-21c728e82922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:48:50.481179 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866098ae-7b00-4716-8b0b-21c728e82922 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:48:50.481456 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866098ae-7b00-4716-8b0b-21c728e82922 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:48:50.481731 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866098ae-7b00-4716-8b0b-21c728e82922 admin admin] PATH_INFO: `/v3/users/87ed2df69d9d4dc58aead0b1e0fbceea` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:48:50.482907 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866098ae-7b00-4716-8b0b-21c728e82922 admin admin] RBAC: Authorizing `identity:delete_user(user_id=87ed2df69d9d4dc58aead0b1e0fbceea)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:48:50.484831 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866098ae-7b00-4716-8b0b-21c728e82922 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:48:50.533591 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-866098ae-7b00-4716-8b0b-21c728e82922 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87ed2df69d9d4dc58aead0b1e0fbceea'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:48:50.540645 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-866098ae-7b00-4716-8b0b-21c728e82922 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87ed2df69d9d4dc58aead0b1e0fbceea'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:48:50.549215 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-866098ae-7b00-4716-8b0b-21c728e82922 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87ed2df69d9d4dc58aead0b1e0fbceea'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:48:50.549880 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-866098ae-7b00-4716-8b0b-21c728e82922 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87ed2df69d9d4dc58aead0b1e0fbceea'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:48:50.550490 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-866098ae-7b00-4716-8b0b-21c728e82922 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87ed2df69d9d4dc58aead0b1e0fbceea'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:48:50.569416 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1194/2412] 10.222.0.89 () {58 vars in 1224 bytes} [Thu Jul 28 17:48:50 2022] DELETE /identity/v3/users/87ed2df69d9d4dc58aead0b1e0fbceea => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:48:50.574490 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d80d88-87c6-4fb5-a005-dd5c6da7b484 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:48:50.574984 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d80d88-87c6-4fb5-a005-dd5c6da7b484 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:48:50.575074 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d80d88-87c6-4fb5-a005-dd5c6da7b484 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:48:50.594332 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-b0d80d88-87c6-4fb5-a005-dd5c6da7b484 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:48:50.595548 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-b0d80d88-87c6-4fb5-a005-dd5c6da7b484 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:48:50.655886 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1219/2413] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:48:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:48:50.665591 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866098ae-7b00-4716-8b0b-21c728e82922 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:48:50.666996 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-866098ae-7b00-4716-8b0b-21c728e82922 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:48:50.724521 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866098ae-7b00-4716-8b0b-21c728e82922 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:48:50.728447 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f71b07a-5d2f-447b-87a1-8a777d48dbac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76817cd541294331b0023069092e8e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:48:50.729061 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f71b07a-5d2f-447b-87a1-8a777d48dbac service neutron] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:48:50.729382 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f71b07a-5d2f-447b-87a1-8a777d48dbac service neutron] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:48:50.729665 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f71b07a-5d2f-447b-87a1-8a777d48dbac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:48:50.738877 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f71b07a-5d2f-447b-87a1-8a777d48dbac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:48:50.740235 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f71b07a-5d2f-447b-87a1-8a777d48dbac service neutron] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:48:50.750568 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1195/2414] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:48:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:48:51.034719 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d80d88-87c6-4fb5-a005-dd5c6da7b484 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:48:51.036144 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-b0d80d88-87c6-4fb5-a005-dd5c6da7b484 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:48:51.048992 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d80d88-87c6-4fb5-a005-dd5c6da7b484 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:48:51.053694 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b450bb0-71a5-48d4-9218-036c195eb188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:48:51.054322 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b450bb0-71a5-48d4-9218-036c195eb188 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:48:51.054600 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b450bb0-71a5-48d4-9218-036c195eb188 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:48:51.054871 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b450bb0-71a5-48d4-9218-036c195eb188 admin admin] PATH_INFO: `/v3/projects/dcc5c66384d546d298795f25eabcc5c9` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:48:51.057101 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b450bb0-71a5-48d4-9218-036c195eb188 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dcc5c66384d546d298795f25eabcc5c9)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:48:51.058963 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b450bb0-71a5-48d4-9218-036c195eb188 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:48:51.106540 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-7b450bb0-71a5-48d4-9218-036c195eb188 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dcc5c66384d546d298795f25eabcc5c9', 'request_id': 'req-7b450bb0-71a5-48d4-9218-036c195eb188'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:48:51.112035 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-7b450bb0-71a5-48d4-9218-036c195eb188 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dcc5c66384d546d298795f25eabcc5c9', 'request_id': 'req-7b450bb0-71a5-48d4-9218-036c195eb188'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:48:51.118167 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-7b450bb0-71a5-48d4-9218-036c195eb188 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dcc5c66384d546d298795f25eabcc5c9', 'request_id': 'req-7b450bb0-71a5-48d4-9218-036c195eb188'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:48:51.131401 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-7b450bb0-71a5-48d4-9218-036c195eb188 admin admin] The token cache is being invalidate because project dcc5c66384d546d298795f25eabcc5c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80324) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 17:48:51.131689 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-7b450bb0-71a5-48d4-9218-036c195eb188 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:48:51.132305 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-7b450bb0-71a5-48d4-9218-036c195eb188 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:48:51.133884 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1220/2415] 10.222.0.89 () {58 vars in 1233 bytes} [Thu Jul 28 17:48:51 2022] DELETE /identity/v3/projects/dcc5c66384d546d298795f25eabcc5c9 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:48:51.138897 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbc9f90-a0cc-403b-9b7a-42adc982039e None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:48:51.139219 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbc9f90-a0cc-403b-9b7a-42adc982039e None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:48:51.139499 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbc9f90-a0cc-403b-9b7a-42adc982039e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:48:51.159634 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-8fbc9f90-a0cc-403b-9b7a-42adc982039e None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:48:51.160852 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-8fbc9f90-a0cc-403b-9b7a-42adc982039e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:48:51.178601 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:48:51.224500 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1196/2416] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:48:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:48:51.288820 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfa40b2-b4e8-4f8e-a57d-919673e7ccdc None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:48:51.289010 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfa40b2-b4e8-4f8e-a57d-919673e7ccdc None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:48:51.289324 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfa40b2-b4e8-4f8e-a57d-919673e7ccdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:48:51.309650 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-fbfa40b2-b4e8-4f8e-a57d-919673e7ccdc None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:48:51.310872 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-fbfa40b2-b4e8-4f8e-a57d-919673e7ccdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:48:51.318828 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1221/2417] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:48:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:48:51.323553 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbc9f90-a0cc-403b-9b7a-42adc982039e None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:48:51.333544 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbc9f90-a0cc-403b-9b7a-42adc982039e None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:48:51.338868 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6551de-5e73-4f36-8c75-eba0694fb26e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:48:51.339480 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6551de-5e73-4f36-8c75-eba0694fb26e admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:48:51.339764 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6551de-5e73-4f36-8c75-eba0694fb26e admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:48:51.340043 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6551de-5e73-4f36-8c75-eba0694fb26e admin admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:48:51.340742 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6551de-5e73-4f36-8c75-eba0694fb26e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:48:51.341052 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6551de-5e73-4f36-8c75-eba0694fb26e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:48:51.342702 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6551de-5e73-4f36-8c75-eba0694fb26e admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:48:51.355000 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1197/2418] 10.222.0.89 () {58 vars in 1153 bytes} [Thu Jul 28 17:48:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:48:51.359514 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfa40b2-b4e8-4f8e-a57d-919673e7ccdc None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:48:51.369210 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfa40b2-b4e8-4f8e-a57d-919673e7ccdc None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:48:51.373857 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff86e0fa-6ea6-427b-9e88-5447e455939a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:48:51.374448 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff86e0fa-6ea6-427b-9e88-5447e455939a admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:48:51.374726 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff86e0fa-6ea6-427b-9e88-5447e455939a admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:48:51.374997 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff86e0fa-6ea6-427b-9e88-5447e455939a admin admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:48:51.375662 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff86e0fa-6ea6-427b-9e88-5447e455939a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:48:51.377595 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff86e0fa-6ea6-427b-9e88-5447e455939a admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:48:51.406647 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1222/2419] 10.222.0.89 () {60 vars in 1153 bytes} [Thu Jul 28 17:48:51 2022] POST /identity/v3/projects => generated 397 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:48:51.410874 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6551de-5e73-4f36-8c75-eba0694fb26e admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:48:51.425424 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6551de-5e73-4f36-8c75-eba0694fb26e admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:48:51.489350 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b1910b-e610-4da2-a7cf-f003f406d27b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:48:51.489968 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b1910b-e610-4da2-a7cf-f003f406d27b admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:48:51.490256 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b1910b-e610-4da2-a7cf-f003f406d27b admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:48:51.490527 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b1910b-e610-4da2-a7cf-f003f406d27b admin admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:48:51.491199 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b1910b-e610-4da2-a7cf-f003f406d27b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:48:51.493099 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b1910b-e610-4da2-a7cf-f003f406d27b admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:48:51.520454 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1198/2420] 10.222.0.89 () {60 vars in 1144 bytes} [Thu Jul 28 17:48:51 2022] POST /identity/v3/users => generated 299 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:48:51.525527 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff86e0fa-6ea6-427b-9e88-5447e455939a admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:48:51.537255 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff86e0fa-6ea6-427b-9e88-5447e455939a admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:48:51.543557 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057db363-cf25-405a-8f83-fd5beb6ab743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:48:51.544140 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057db363-cf25-405a-8f83-fd5beb6ab743 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:48:51.544426 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057db363-cf25-405a-8f83-fd5beb6ab743 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:48:51.544754 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057db363-cf25-405a-8f83-fd5beb6ab743 admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:48:51.545545 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057db363-cf25-405a-8f83-fd5beb6ab743 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:48:51.547222 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057db363-cf25-405a-8f83-fd5beb6ab743 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:48:51.558283 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1223/2421] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:48:51 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 17:48:51.566439 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b1910b-e610-4da2-a7cf-f003f406d27b admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:48:51.578145 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b1910b-e610-4da2-a7cf-f003f406d27b admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:48:51.583155 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e492a8d-b79e-47cc-9d36-4478351ef232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:48:51.583850 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e492a8d-b79e-47cc-9d36-4478351ef232 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:48:51.584128 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e492a8d-b79e-47cc-9d36-4478351ef232 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:48:51.584401 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e492a8d-b79e-47cc-9d36-4478351ef232 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:48:51.586785 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e492a8d-b79e-47cc-9d36-4478351ef232 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:48:51.586785 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e492a8d-b79e-47cc-9d36-4478351ef232 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:48:51.598619 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1199/2422] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:48:51 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:48:51.602883 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057db363-cf25-405a-8f83-fd5beb6ab743 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:48:51.614420 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057db363-cf25-405a-8f83-fd5beb6ab743 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:48:51.619815 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865ff9d8-e7af-43fd-b630-a85768be35bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:48:51.620491 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865ff9d8-e7af-43fd-b630-a85768be35bb admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:48:51.620775 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865ff9d8-e7af-43fd-b630-a85768be35bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:48:51.621050 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865ff9d8-e7af-43fd-b630-a85768be35bb admin admin] PATH_INFO: `/v3/projects/32fc268c2a6d48bbaeeb8c79f72d7159/users/8b04e9f607494075b8688ae8c056a894/roles/11f8731756fe4caca13efa61b71b11c5` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:48:51.638700 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865ff9d8-e7af-43fd-b630-a85768be35bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=32fc268c2a6d48bbaeeb8c79f72d7159, user_id=8b04e9f607494075b8688ae8c056a894, role_id=11f8731756fe4caca13efa61b71b11c5)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:48:51.641700 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865ff9d8-e7af-43fd-b630-a85768be35bb admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:48:51.671214 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1224/2423] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:48:51 2022] PUT /identity/v3/projects/32fc268c2a6d48bbaeeb8c79f72d7159/users/8b04e9f607494075b8688ae8c056a894/roles/11f8731756fe4caca13efa61b71b11c5 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:48:51.682007 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8f3354-1c75-498d-a5c9-a7b0553ab003 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:48:51.682289 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8f3354-1c75-498d-a5c9-a7b0553ab003 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:48:51.682559 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8f3354-1c75-498d-a5c9-a7b0553ab003 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:48:51.714588 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-2f8f3354-1c75-498d-a5c9-a7b0553ab003 None None] MFA Rules not processed for user `8b04e9f607494075b8688ae8c056a894`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:48:51.715790 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-2f8f3354-1c75-498d-a5c9-a7b0553ab003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:48:51.783250 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1200/2424] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:48:51 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:48:51.794785 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865ff9d8-e7af-43fd-b630-a85768be35bb admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:48:51.796431 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-865ff9d8-e7af-43fd-b630-a85768be35bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:48:51.818437 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:48:51.857096 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865ff9d8-e7af-43fd-b630-a85768be35bb admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:48:51.860859 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bd0af6-d033-4507-b448-a08e6b410352 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96d512d0c84240918dae6c4a3e8b7b79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:48:51.861494 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bd0af6-d033-4507-b448-a08e6b410352 service cinder] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:48:51.861775 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bd0af6-d033-4507-b448-a08e6b410352 service cinder] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:48:51.862050 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bd0af6-d033-4507-b448-a08e6b410352 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:48:51.871087 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bd0af6-d033-4507-b448-a08e6b410352 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:48:51.872603 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bd0af6-d033-4507-b448-a08e6b410352 service cinder] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:48:51.881914 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1225/2425] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:48:51 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:48:59.981345 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8f3354-1c75-498d-a5c9-a7b0553ab003 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:49:00.009484 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:49:00.058178 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8f3354-1c75-498d-a5c9-a7b0553ab003 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:49:00.063209 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d25632-bd8e-4935-90b7-230fd6feafdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:49:00.063863 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d25632-bd8e-4935-90b7-230fd6feafdf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:49:00.064143 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d25632-bd8e-4935-90b7-230fd6feafdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:49:00.064417 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d25632-bd8e-4935-90b7-230fd6feafdf admin admin] PATH_INFO: `/v3/users/8b04e9f607494075b8688ae8c056a894` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:49:00.065632 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d25632-bd8e-4935-90b7-230fd6feafdf admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b04e9f607494075b8688ae8c056a894)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:49:00.067636 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d25632-bd8e-4935-90b7-230fd6feafdf admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:49:00.145678 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-a9d25632-bd8e-4935-90b7-230fd6feafdf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b04e9f607494075b8688ae8c056a894'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:49:00.157568 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-a9d25632-bd8e-4935-90b7-230fd6feafdf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b04e9f607494075b8688ae8c056a894'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:49:00.173591 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-a9d25632-bd8e-4935-90b7-230fd6feafdf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b04e9f607494075b8688ae8c056a894'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:49:00.174471 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-a9d25632-bd8e-4935-90b7-230fd6feafdf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b04e9f607494075b8688ae8c056a894'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:49:00.175126 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-a9d25632-bd8e-4935-90b7-230fd6feafdf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b04e9f607494075b8688ae8c056a894'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:49:00.201580 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1201/2426] 10.222.0.89 () {58 vars in 1224 bytes} [Thu Jul 28 17:48:59 2022] DELETE /identity/v3/users/8b04e9f607494075b8688ae8c056a894 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:49:00.206373 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e923f575-08c3-4525-8a4c-709a2f5b2098 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:49:00.206928 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e923f575-08c3-4525-8a4c-709a2f5b2098 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:49:00.206928 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e923f575-08c3-4525-8a4c-709a2f5b2098 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:49:00.227209 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-e923f575-08c3-4525-8a4c-709a2f5b2098 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:49:00.229064 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-e923f575-08c3-4525-8a4c-709a2f5b2098 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:49:00.307789 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1226/2427] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:49:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:49:00.317806 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d25632-bd8e-4935-90b7-230fd6feafdf admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:49:00.319284 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-a9d25632-bd8e-4935-90b7-230fd6feafdf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:49:00.377227 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d25632-bd8e-4935-90b7-230fd6feafdf admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:49:00.380764 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3f03f8-258a-4d7b-9927-6346e5fe3957 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76817cd541294331b0023069092e8e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:49:00.381408 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3f03f8-258a-4d7b-9927-6346e5fe3957 service neutron] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:49:00.381686 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3f03f8-258a-4d7b-9927-6346e5fe3957 service neutron] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:49:00.381954 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3f03f8-258a-4d7b-9927-6346e5fe3957 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:49:00.391596 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3f03f8-258a-4d7b-9927-6346e5fe3957 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:49:00.392993 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3f03f8-258a-4d7b-9927-6346e5fe3957 service neutron] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:49:00.403313 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1202/2428] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:49:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:49:00.680224 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e923f575-08c3-4525-8a4c-709a2f5b2098 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:49:00.681979 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-e923f575-08c3-4525-8a4c-709a2f5b2098 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:49:00.694382 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e923f575-08c3-4525-8a4c-709a2f5b2098 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:49:00.700271 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d84919-ae6d-4d39-8a3f-9a1d000eb6a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:49:00.700596 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d84919-ae6d-4d39-8a3f-9a1d000eb6a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:49:00.700877 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d84919-ae6d-4d39-8a3f-9a1d000eb6a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:49:00.701151 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d84919-ae6d-4d39-8a3f-9a1d000eb6a5 admin admin] PATH_INFO: `/v3/projects/32fc268c2a6d48bbaeeb8c79f72d7159` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:49:00.703218 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d84919-ae6d-4d39-8a3f-9a1d000eb6a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=32fc268c2a6d48bbaeeb8c79f72d7159)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:49:00.705391 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d84919-ae6d-4d39-8a3f-9a1d000eb6a5 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:49:00.757773 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-d2d84919-ae6d-4d39-8a3f-9a1d000eb6a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32fc268c2a6d48bbaeeb8c79f72d7159', 'request_id': 'req-d2d84919-ae6d-4d39-8a3f-9a1d000eb6a5'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:49:00.764482 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-d2d84919-ae6d-4d39-8a3f-9a1d000eb6a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32fc268c2a6d48bbaeeb8c79f72d7159', 'request_id': 'req-d2d84919-ae6d-4d39-8a3f-9a1d000eb6a5'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:49:00.771665 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-d2d84919-ae6d-4d39-8a3f-9a1d000eb6a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32fc268c2a6d48bbaeeb8c79f72d7159', 'request_id': 'req-d2d84919-ae6d-4d39-8a3f-9a1d000eb6a5'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:49:00.785791 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-d2d84919-ae6d-4d39-8a3f-9a1d000eb6a5 admin admin] The token cache is being invalidate because project 32fc268c2a6d48bbaeeb8c79f72d7159 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80324) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 17:49:00.786086 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-d2d84919-ae6d-4d39-8a3f-9a1d000eb6a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:49:00.786684 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-d2d84919-ae6d-4d39-8a3f-9a1d000eb6a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:49:00.788199 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1227/2429] 10.222.0.89 () {58 vars in 1233 bytes} [Thu Jul 28 17:49:00 2022] DELETE /identity/v3/projects/32fc268c2a6d48bbaeeb8c79f72d7159 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:49:10.398314 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=71,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:49:10.782757 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=71,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:49:14.350159 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a2e42a-f043-4e62-ac52-c2045850b7de None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:49:14.350726 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a2e42a-f043-4e62-ac52-c2045850b7de None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:49:14.350726 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a2e42a-f043-4e62-ac52-c2045850b7de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:49:14.370397 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-27a2e42a-f043-4e62-ac52-c2045850b7de None None] MFA Rules not processed for user `14327b8153ab4de782d4bcfef7681fc1`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:49:14.371624 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-27a2e42a-f043-4e62-ac52-c2045850b7de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:49:14.460267 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1203/2430] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:49:14 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:49:16.173582 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a327b2b-91d4-4696-95e1-d2fa9b25f5bb None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:49:16.175163 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a327b2b-91d4-4696-95e1-d2fa9b25f5bb None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:49:16.176399 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a327b2b-91d4-4696-95e1-d2fa9b25f5bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:49:16.196361 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-1a327b2b-91d4-4696-95e1-d2fa9b25f5bb None None] MFA Rules not processed for user `c44387d13a694ee6bb47a13906003330`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:49:16.198324 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-1a327b2b-91d4-4696-95e1-d2fa9b25f5bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:49:16.262432 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1228/2431] 10.222.0.89 () {58 vars in 967 bytes} [Thu Jul 28 17:49:16 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:49:16.313894 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a2e42a-f043-4e62-ac52-c2045850b7de None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:49:16.315513 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-27a2e42a-f043-4e62-ac52-c2045850b7de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:49:16.404210 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a2e42a-f043-4e62-ac52-c2045850b7de None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:49:16.412930 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb13c3da-fc79-4d47-83f7-7dea533ee8cf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c59473b01bf4433ba3b111114fc048b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:49:16.413213 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb13c3da-fc79-4d47-83f7-7dea533ee8cf service swift] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:49:16.413388 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb13c3da-fc79-4d47-83f7-7dea533ee8cf service swift] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:49:16.413629 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb13c3da-fc79-4d47-83f7-7dea533ee8cf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:49:16.423918 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb13c3da-fc79-4d47-83f7-7dea533ee8cf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:49:16.425375 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb13c3da-fc79-4d47-83f7-7dea533ee8cf service swift] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:49:16.438851 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1204/2432] 10.222.0.89 () {60 vars in 1384 bytes} [Thu Jul 28 17:49:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 17:49:17.909173 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a327b2b-91d4-4696-95e1-d2fa9b25f5bb None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:49:17.910880 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-1a327b2b-91d4-4696-95e1-d2fa9b25f5bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:49:17.970641 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a327b2b-91d4-4696-95e1-d2fa9b25f5bb None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:49:17.973501 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e46158-57fb-4740-9c67-68fd9323d25b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76817cd541294331b0023069092e8e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:49:17.974109 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e46158-57fb-4740-9c67-68fd9323d25b service neutron] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:49:17.974388 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e46158-57fb-4740-9c67-68fd9323d25b service neutron] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:49:17.974658 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e46158-57fb-4740-9c67-68fd9323d25b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:49:17.976437 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-f7e46158-57fb-4740-9c67-68fd9323d25b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:49:18.039044 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e46158-57fb-4740-9c67-68fd9323d25b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:49:18.040476 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e46158-57fb-4740-9c67-68fd9323d25b service neutron] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:49:18.050570 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1229/2433] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:49:17 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:49:24.637540 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb13c3da-fc79-4d47-83f7-7dea533ee8cf service swift] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:49:24.647263 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb13c3da-fc79-4d47-83f7-7dea533ee8cf service swift] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:49:24.651585 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc599a55-9ff2-42e5-8b69-fc1742c66de5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76817cd541294331b0023069092e8e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:49:24.652188 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc599a55-9ff2-42e5-8b69-fc1742c66de5 service neutron] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:49:24.652466 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc599a55-9ff2-42e5-8b69-fc1742c66de5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:49:24.652764 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc599a55-9ff2-42e5-8b69-fc1742c66de5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:49:24.654630 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-cc599a55-9ff2-42e5-8b69-fc1742c66de5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:49:24.723855 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc599a55-9ff2-42e5-8b69-fc1742c66de5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:49:24.725310 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc599a55-9ff2-42e5-8b69-fc1742c66de5 service neutron] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:49:24.736604 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1205/2434] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:49:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:49:25.925244 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e46158-57fb-4740-9c67-68fd9323d25b service neutron] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:49:25.926774 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-f7e46158-57fb-4740-9c67-68fd9323d25b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:49:25.938918 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e46158-57fb-4740-9c67-68fd9323d25b service neutron] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:49:25.943108 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e233659-2dcf-43e3-955a-6a03a620cd17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76817cd541294331b0023069092e8e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:49:25.943718 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e233659-2dcf-43e3-955a-6a03a620cd17 service neutron] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:49:25.943997 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e233659-2dcf-43e3-955a-6a03a620cd17 service neutron] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:49:25.944270 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e233659-2dcf-43e3-955a-6a03a620cd17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:49:25.946042 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-4e233659-2dcf-43e3-955a-6a03a620cd17 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:49:25.961489 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e233659-2dcf-43e3-955a-6a03a620cd17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:49:25.962993 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e233659-2dcf-43e3-955a-6a03a620cd17 service neutron] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:49:25.977065 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1230/2435] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:49:25 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:49:34.731526 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=91 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:49:35.971737 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=81 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:50:00.941931 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc599a55-9ff2-42e5-8b69-fc1742c66de5 service neutron] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:50:00.943546 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-cc599a55-9ff2-42e5-8b69-fc1742c66de5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:50:00.957259 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc599a55-9ff2-42e5-8b69-fc1742c66de5 service neutron] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:50:00.962702 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3518254-84f2-4c94-ba69-3d838e8c04eb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd10b49b29df94404a71f43273839c252', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:50:00.963309 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3518254-84f2-4c94-ba69-3d838e8c04eb service nova] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:50:00.963583 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3518254-84f2-4c94-ba69-3d838e8c04eb service nova] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:50:00.963852 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3518254-84f2-4c94-ba69-3d838e8c04eb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:50:00.965580 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-f3518254-84f2-4c94-ba69-3d838e8c04eb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:50:01.025268 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3518254-84f2-4c94-ba69-3d838e8c04eb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:50:01.026852 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3518254-84f2-4c94-ba69-3d838e8c04eb service nova] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:50:01.037504 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1206/2436] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:50:00 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:50:04.629910 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e233659-2dcf-43e3-955a-6a03a620cd17 service neutron] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:50:04.631319 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-4e233659-2dcf-43e3-955a-6a03a620cd17 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:50:04.688863 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e233659-2dcf-43e3-955a-6a03a620cd17 service neutron] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:50:04.692498 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3fb2e7-6ba5-4ddc-938c-3323b0c1c614 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96d512d0c84240918dae6c4a3e8b7b79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:50:04.693106 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3fb2e7-6ba5-4ddc-938c-3323b0c1c614 service cinder] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:50:04.693414 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3fb2e7-6ba5-4ddc-938c-3323b0c1c614 service cinder] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:50:04.693686 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3fb2e7-6ba5-4ddc-938c-3323b0c1c614 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:50:04.695440 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-bc3fb2e7-6ba5-4ddc-938c-3323b0c1c614 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:50:04.706381 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3fb2e7-6ba5-4ddc-938c-3323b0c1c614 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:50:04.707792 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3fb2e7-6ba5-4ddc-938c-3323b0c1c614 service cinder] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:50:04.716725 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1231/2437] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:50:04 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:50:07.383710 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3518254-84f2-4c94-ba69-3d838e8c04eb service nova] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:50:07.398651 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3518254-84f2-4c94-ba69-3d838e8c04eb service nova] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:50:07.406230 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf334a0-48d1-40a2-af8a-ae40ecdeaa9a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd10b49b29df94404a71f43273839c252', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:50:07.406954 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf334a0-48d1-40a2-af8a-ae40ecdeaa9a service nova] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:50:07.407430 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf334a0-48d1-40a2-af8a-ae40ecdeaa9a service nova] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:50:07.407711 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf334a0-48d1-40a2-af8a-ae40ecdeaa9a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:50:07.410875 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-6bf334a0-48d1-40a2-af8a-ae40ecdeaa9a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:50:07.432932 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf334a0-48d1-40a2-af8a-ae40ecdeaa9a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:50:07.434898 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf334a0-48d1-40a2-af8a-ae40ecdeaa9a service nova] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:50:07.450596 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1207/2438] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:50:07 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:50:14.712601 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=26 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:50:15.062555 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ac0bb9-e668-4db9-ba7d-68aeec55d8e5 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:50:15.063125 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ac0bb9-e668-4db9-ba7d-68aeec55d8e5 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:50:15.063629 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ac0bb9-e668-4db9-ba7d-68aeec55d8e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:50:15.085005 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-72ac0bb9-e668-4db9-ba7d-68aeec55d8e5 None None] MFA Rules not processed for user `0b97c60c0360466fb878e1e9141e1f3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:50:15.086413 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-72ac0bb9-e668-4db9-ba7d-68aeec55d8e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:50:15.094206 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1232/2439] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:50:15 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:50:15.403346 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf334a0-48d1-40a2-af8a-ae40ecdeaa9a service nova] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:50:15.414096 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf334a0-48d1-40a2-af8a-ae40ecdeaa9a service nova] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:50:15.417753 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b528be-0234-469c-b36e-425d8f291864 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76817cd541294331b0023069092e8e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:50:15.418351 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b528be-0234-469c-b36e-425d8f291864 service neutron] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:50:15.418627 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b528be-0234-469c-b36e-425d8f291864 service neutron] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:50:15.418895 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b528be-0234-469c-b36e-425d8f291864 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:50:15.420657 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-19b528be-0234-469c-b36e-425d8f291864 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:50:15.430195 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b528be-0234-469c-b36e-425d8f291864 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:50:15.431603 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b528be-0234-469c-b36e-425d8f291864 service neutron] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:50:15.439955 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1208/2440] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:50:15 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:50:15.982506 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ac0bb9-e668-4db9-ba7d-68aeec55d8e5 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:50:15.985169 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-72ac0bb9-e668-4db9-ba7d-68aeec55d8e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:50:16.061700 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ac0bb9-e668-4db9-ba7d-68aeec55d8e5 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:50:16.066797 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385e4174-9c90-46aa-8e12-ac4a721276e8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4bd290018394b0d955cd376ce35b77f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:50:16.067413 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385e4174-9c90-46aa-8e12-ac4a721276e8 service placement] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:50:16.067701 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385e4174-9c90-46aa-8e12-ac4a721276e8 service placement] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:50:16.068024 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385e4174-9c90-46aa-8e12-ac4a721276e8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:50:16.069802 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-385e4174-9c90-46aa-8e12-ac4a721276e8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:50:16.081745 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385e4174-9c90-46aa-8e12-ac4a721276e8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:50:16.083250 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385e4174-9c90-46aa-8e12-ac4a721276e8 service placement] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:50:16.093872 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1233/2441] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:50:15 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:50:17.843524 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b528be-0234-469c-b36e-425d8f291864 service neutron] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:50:17.844956 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-19b528be-0234-469c-b36e-425d8f291864 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:50:17.856252 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b528be-0234-469c-b36e-425d8f291864 service neutron] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:50:17.860909 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8ee9af-7d96-4a78-8be1-a9435e2d98d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96d512d0c84240918dae6c4a3e8b7b79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:50:17.861558 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8ee9af-7d96-4a78-8be1-a9435e2d98d7 service cinder] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:50:17.861839 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8ee9af-7d96-4a78-8be1-a9435e2d98d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:50:17.862112 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8ee9af-7d96-4a78-8be1-a9435e2d98d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:50:17.864037 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-bd8ee9af-7d96-4a78-8be1-a9435e2d98d7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:50:17.927972 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8ee9af-7d96-4a78-8be1-a9435e2d98d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:50:17.929440 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8ee9af-7d96-4a78-8be1-a9435e2d98d7 service cinder] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:50:17.942669 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1209/2442] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:50:17 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:50:18.975485 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385e4174-9c90-46aa-8e12-ac4a721276e8 service placement] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:50:18.987468 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385e4174-9c90-46aa-8e12-ac4a721276e8 service placement] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:50:18.991670 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0464d675-5431-4e30-b515-aba17a12aa4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76817cd541294331b0023069092e8e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:50:18.992276 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0464d675-5431-4e30-b515-aba17a12aa4b service neutron] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:50:18.992555 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0464d675-5431-4e30-b515-aba17a12aa4b service neutron] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:50:18.992825 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0464d675-5431-4e30-b515-aba17a12aa4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:50:18.994624 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-0464d675-5431-4e30-b515-aba17a12aa4b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:50:19.011853 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0464d675-5431-4e30-b515-aba17a12aa4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:50:19.013337 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0464d675-5431-4e30-b515-aba17a12aa4b service neutron] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:50:19.025728 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1234/2443] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:50:18 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:50:27.934664 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=68 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:50:28.317251 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8ee9af-7d96-4a78-8be1-a9435e2d98d7 service cinder] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:50:28.326629 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8ee9af-7d96-4a78-8be1-a9435e2d98d7 service cinder] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:50:28.331244 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cecf86-8d47-45ca-98a3-2335c50b7854 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76817cd541294331b0023069092e8e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:50:28.331864 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cecf86-8d47-45ca-98a3-2335c50b7854 service neutron] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:50:28.332150 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cecf86-8d47-45ca-98a3-2335c50b7854 service neutron] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:50:28.332426 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cecf86-8d47-45ca-98a3-2335c50b7854 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:50:28.334250 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-57cecf86-8d47-45ca-98a3-2335c50b7854 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:50:28.412573 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cecf86-8d47-45ca-98a3-2335c50b7854 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:50:28.414317 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cecf86-8d47-45ca-98a3-2335c50b7854 service neutron] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:50:28.426420 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1210/2444] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:50:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:50:28.535717 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0464d675-5431-4e30-b515-aba17a12aa4b service neutron] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:50:28.551476 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0464d675-5431-4e30-b515-aba17a12aa4b service neutron] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:50:28.558035 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaee046-e53e-4a41-bf0c-19c8741db461 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd10b49b29df94404a71f43273839c252', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:50:28.558813 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaee046-e53e-4a41-bf0c-19c8741db461 service nova] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:50:28.559229 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaee046-e53e-4a41-bf0c-19c8741db461 service nova] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:50:28.559614 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaee046-e53e-4a41-bf0c-19c8741db461 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:50:28.561616 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-6eaee046-e53e-4a41-bf0c-19c8741db461 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:50:28.576932 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eaee046-e53e-4a41-bf0c-19c8741db461 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:50:28.579291 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eaee046-e53e-4a41-bf0c-19c8741db461 service nova] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:50:28.592233 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1235/2445] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:50:28 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:50:29.371086 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cecf86-8d47-45ca-98a3-2335c50b7854 service neutron] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:50:29.387481 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cecf86-8d47-45ca-98a3-2335c50b7854 service neutron] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:50:29.400316 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e4a12d-3512-433e-80a7-584e79bab8f3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd10b49b29df94404a71f43273839c252', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:50:29.401287 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e4a12d-3512-433e-80a7-584e79bab8f3 service nova] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:50:29.401769 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e4a12d-3512-433e-80a7-584e79bab8f3 service nova] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:50:29.402332 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e4a12d-3512-433e-80a7-584e79bab8f3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:50:29.408818 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-c8e4a12d-3512-433e-80a7-584e79bab8f3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:50:29.428632 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e4a12d-3512-433e-80a7-584e79bab8f3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:50:29.431176 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e4a12d-3512-433e-80a7-584e79bab8f3 service nova] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:50:29.447591 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1211/2446] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:50:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:50:30.363985 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e909a5d2-4d20-4d11-aa97-4f1fcd8739d9 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:50:30.364862 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e909a5d2-4d20-4d11-aa97-4f1fcd8739d9 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:50:30.365742 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e909a5d2-4d20-4d11-aa97-4f1fcd8739d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:50:30.389613 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-e909a5d2-4d20-4d11-aa97-4f1fcd8739d9 None None] MFA Rules not processed for user `22479a66b0f34038b144f4def1aa3f43`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:50:30.391335 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-e909a5d2-4d20-4d11-aa97-4f1fcd8739d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:50:30.398623 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1236/2447] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:50:30 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:50:31.126239 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e4a12d-3512-433e-80a7-584e79bab8f3 service nova] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:50:31.137350 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e4a12d-3512-433e-80a7-584e79bab8f3 service nova] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:50:31.142668 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da918e70-ff30-444e-9ea4-e4aab067c5a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76817cd541294331b0023069092e8e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:50:31.143409 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da918e70-ff30-444e-9ea4-e4aab067c5a8 service neutron] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:50:31.144017 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da918e70-ff30-444e-9ea4-e4aab067c5a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:50:31.144758 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da918e70-ff30-444e-9ea4-e4aab067c5a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:50:31.146967 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-da918e70-ff30-444e-9ea4-e4aab067c5a8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:50:31.160142 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da918e70-ff30-444e-9ea4-e4aab067c5a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:50:31.161642 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da918e70-ff30-444e-9ea4-e4aab067c5a8 service neutron] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:50:31.172077 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1212/2448] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:50:31 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:50:35.048839 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e909a5d2-4d20-4d11-aa97-4f1fcd8739d9 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:50:35.050573 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-e909a5d2-4d20-4d11-aa97-4f1fcd8739d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:50:35.067883 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e909a5d2-4d20-4d11-aa97-4f1fcd8739d9 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:50:35.072699 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1a6a41-0e18-42ef-9974-afba001ec28f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:50:35.073416 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1a6a41-0e18-42ef-9974-afba001ec28f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:50:35.073856 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1a6a41-0e18-42ef-9974-afba001ec28f admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:50:35.074139 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1a6a41-0e18-42ef-9974-afba001ec28f admin admin] PATH_INFO: `/v3/users/14327b8153ab4de782d4bcfef7681fc1` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:50:35.075443 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1a6a41-0e18-42ef-9974-afba001ec28f admin admin] RBAC: Authorizing `identity:delete_user(user_id=14327b8153ab4de782d4bcfef7681fc1)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:50:35.078378 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1a6a41-0e18-42ef-9974-afba001ec28f admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:50:35.134788 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-2b1a6a41-0e18-42ef-9974-afba001ec28f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14327b8153ab4de782d4bcfef7681fc1'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:50:35.142594 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-2b1a6a41-0e18-42ef-9974-afba001ec28f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14327b8153ab4de782d4bcfef7681fc1'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:50:35.143278 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-2b1a6a41-0e18-42ef-9974-afba001ec28f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14327b8153ab4de782d4bcfef7681fc1'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:50:35.159683 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-2b1a6a41-0e18-42ef-9974-afba001ec28f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14327b8153ab4de782d4bcfef7681fc1'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:50:35.174296 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-2b1a6a41-0e18-42ef-9974-afba001ec28f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14327b8153ab4de782d4bcfef7681fc1'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:50:35.182894 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1237/2449] 10.222.0.89 () {58 vars in 1224 bytes} [Thu Jul 28 17:50:35 2022] DELETE /identity/v3/users/14327b8153ab4de782d4bcfef7681fc1 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:50:35.370690 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da918e70-ff30-444e-9ea4-e4aab067c5a8 service neutron] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:50:35.386338 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da918e70-ff30-444e-9ea4-e4aab067c5a8 service neutron] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:50:35.464762 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122fb3c0-03e0-4ba8-a2c8-1f15fef365f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:50:35.465456 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122fb3c0-03e0-4ba8-a2c8-1f15fef365f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:50:35.465739 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122fb3c0-03e0-4ba8-a2c8-1f15fef365f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:50:35.466016 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122fb3c0-03e0-4ba8-a2c8-1f15fef365f8 admin admin] PATH_INFO: `/v3/projects/f5332e245b754728b04af219fdb6ca93` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:50:35.467870 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122fb3c0-03e0-4ba8-a2c8-1f15fef365f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5332e245b754728b04af219fdb6ca93)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:50:35.470484 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122fb3c0-03e0-4ba8-a2c8-1f15fef365f8 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:50:35.527421 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-122fb3c0-03e0-4ba8-a2c8-1f15fef365f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5332e245b754728b04af219fdb6ca93', 'request_id': 'req-122fb3c0-03e0-4ba8-a2c8-1f15fef365f8'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:50:35.535245 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-122fb3c0-03e0-4ba8-a2c8-1f15fef365f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5332e245b754728b04af219fdb6ca93', 'request_id': 'req-122fb3c0-03e0-4ba8-a2c8-1f15fef365f8'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:50:35.540967 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-122fb3c0-03e0-4ba8-a2c8-1f15fef365f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5332e245b754728b04af219fdb6ca93', 'request_id': 'req-122fb3c0-03e0-4ba8-a2c8-1f15fef365f8'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:50:35.552727 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:50:35.556980 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-122fb3c0-03e0-4ba8-a2c8-1f15fef365f8 admin admin] The token cache is being invalidate because project f5332e245b754728b04af219fdb6ca93 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80325) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 17:50:35.557629 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-122fb3c0-03e0-4ba8-a2c8-1f15fef365f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:50:35.558319 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-122fb3c0-03e0-4ba8-a2c8-1f15fef365f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:50:35.559917 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1213/2450] 10.222.0.89 () {58 vars in 1233 bytes} [Thu Jul 28 17:50:35 2022] DELETE /identity/v3/projects/f5332e245b754728b04af219fdb6ca93 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:50:35.565262 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f075f17-786a-48cf-ad9a-846715cdee33 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:50:35.565642 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f075f17-786a-48cf-ad9a-846715cdee33 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:50:35.566003 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f075f17-786a-48cf-ad9a-846715cdee33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:50:35.586050 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-4f075f17-786a-48cf-ad9a-846715cdee33 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:50:35.587256 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-4f075f17-786a-48cf-ad9a-846715cdee33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:50:35.604191 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:50:35.652562 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1238/2451] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:50:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:50:35.662100 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a07d68-432c-416d-8895-74a7b6cfd5f7 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:50:35.662392 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a07d68-432c-416d-8895-74a7b6cfd5f7 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:50:35.662673 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a07d68-432c-416d-8895-74a7b6cfd5f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:50:35.682875 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-d8a07d68-432c-416d-8895-74a7b6cfd5f7 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:50:35.684105 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-d8a07d68-432c-416d-8895-74a7b6cfd5f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:50:35.692218 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1214/2452] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:50:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:50:35.696474 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f075f17-786a-48cf-ad9a-846715cdee33 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:50:35.706733 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f075f17-786a-48cf-ad9a-846715cdee33 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:50:35.711250 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cfa047-2d13-48ae-8e48-0a00cc14c1f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:50:35.711843 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cfa047-2d13-48ae-8e48-0a00cc14c1f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:50:35.712116 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cfa047-2d13-48ae-8e48-0a00cc14c1f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:50:35.712384 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cfa047-2d13-48ae-8e48-0a00cc14c1f8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:50:35.713063 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cfa047-2d13-48ae-8e48-0a00cc14c1f8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:50:35.713390 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cfa047-2d13-48ae-8e48-0a00cc14c1f8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:50:35.714983 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cfa047-2d13-48ae-8e48-0a00cc14c1f8 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:50:35.727509 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1239/2453] 10.222.0.89 () {58 vars in 1153 bytes} [Thu Jul 28 17:50:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:50:35.731684 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a07d68-432c-416d-8895-74a7b6cfd5f7 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:50:35.741518 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a07d68-432c-416d-8895-74a7b6cfd5f7 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:50:35.746112 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c94f7b6-fb3a-464f-b4ce-64ecbfea021d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:50:35.746706 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c94f7b6-fb3a-464f-b4ce-64ecbfea021d admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:50:35.746981 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c94f7b6-fb3a-464f-b4ce-64ecbfea021d admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:50:35.747250 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c94f7b6-fb3a-464f-b4ce-64ecbfea021d admin admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:50:35.747898 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c94f7b6-fb3a-464f-b4ce-64ecbfea021d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:50:35.750334 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c94f7b6-fb3a-464f-b4ce-64ecbfea021d admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:50:35.778157 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1215/2454] 10.222.0.89 () {60 vars in 1153 bytes} [Thu Jul 28 17:50:35 2022] POST /identity/v3/projects => generated 383 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:50:35.783195 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cfa047-2d13-48ae-8e48-0a00cc14c1f8 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:50:35.793636 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cfa047-2d13-48ae-8e48-0a00cc14c1f8 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:50:35.851511 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aed8bfd-acd7-47e8-b126-f07d6ed67c8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:50:35.852121 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aed8bfd-acd7-47e8-b126-f07d6ed67c8a admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:50:35.852399 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aed8bfd-acd7-47e8-b126-f07d6ed67c8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:50:35.852676 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aed8bfd-acd7-47e8-b126-f07d6ed67c8a admin admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:50:35.853382 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aed8bfd-acd7-47e8-b126-f07d6ed67c8a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:50:35.855320 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aed8bfd-acd7-47e8-b126-f07d6ed67c8a admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:50:35.876895 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1240/2455] 10.222.0.89 () {60 vars in 1144 bytes} [Thu Jul 28 17:50:35 2022] POST /identity/v3/users => generated 292 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:50:35.881084 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c94f7b6-fb3a-464f-b4ce-64ecbfea021d admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:50:35.892572 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c94f7b6-fb3a-464f-b4ce-64ecbfea021d admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:50:35.897673 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc23a0e0-b025-48ab-b29f-d440b994ff59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:50:35.898287 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc23a0e0-b025-48ab-b29f-d440b994ff59 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:50:35.898571 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc23a0e0-b025-48ab-b29f-d440b994ff59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:50:35.898845 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc23a0e0-b025-48ab-b29f-d440b994ff59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:50:35.899544 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc23a0e0-b025-48ab-b29f-d440b994ff59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:50:35.901176 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc23a0e0-b025-48ab-b29f-d440b994ff59 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:50:35.912536 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1216/2456] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:50:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:50:35.916791 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aed8bfd-acd7-47e8-b126-f07d6ed67c8a admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:50:35.927030 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aed8bfd-acd7-47e8-b126-f07d6ed67c8a admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:50:35.933663 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a458fd-fa1a-4cc4-a66c-6e81684deece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:50:35.934276 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a458fd-fa1a-4cc4-a66c-6e81684deece admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:50:35.934557 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a458fd-fa1a-4cc4-a66c-6e81684deece admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:50:35.934830 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a458fd-fa1a-4cc4-a66c-6e81684deece admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:50:35.935523 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a458fd-fa1a-4cc4-a66c-6e81684deece admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:50:35.937304 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a458fd-fa1a-4cc4-a66c-6e81684deece admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:50:35.948246 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1241/2457] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:50:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:50:35.952330 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc23a0e0-b025-48ab-b29f-d440b994ff59 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:50:35.964151 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc23a0e0-b025-48ab-b29f-d440b994ff59 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:50:35.970166 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87be535e-6853-43e9-9749-c413ec7404e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:50:35.970844 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87be535e-6853-43e9-9749-c413ec7404e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:50:35.971173 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87be535e-6853-43e9-9749-c413ec7404e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:50:35.971499 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87be535e-6853-43e9-9749-c413ec7404e7 admin admin] PATH_INFO: `/v3/projects/72a9ad9df4f84c658e9964f1a0c0b282/users/39ff02cba2fd4e7b9fb27b98b9daa82e/roles/11f8731756fe4caca13efa61b71b11c5` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:50:35.987810 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87be535e-6853-43e9-9749-c413ec7404e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=72a9ad9df4f84c658e9964f1a0c0b282, user_id=39ff02cba2fd4e7b9fb27b98b9daa82e, role_id=11f8731756fe4caca13efa61b71b11c5)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:50:35.990608 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87be535e-6853-43e9-9749-c413ec7404e7 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:50:36.009894 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1217/2458] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:50:35 2022] PUT /identity/v3/projects/72a9ad9df4f84c658e9964f1a0c0b282/users/39ff02cba2fd4e7b9fb27b98b9daa82e/roles/11f8731756fe4caca13efa61b71b11c5 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:50:36.011894 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fecef3-24b9-44da-acd4-1e966e77dd43 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:50:36.012180 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fecef3-24b9-44da-acd4-1e966e77dd43 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:50:36.012472 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fecef3-24b9-44da-acd4-1e966e77dd43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:50:36.033202 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-45fecef3-24b9-44da-acd4-1e966e77dd43 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:50:36.034364 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-45fecef3-24b9-44da-acd4-1e966e77dd43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:50:36.103730 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1242/2459] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:50:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:50:36.112995 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87be535e-6853-43e9-9749-c413ec7404e7 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:50:36.114873 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-87be535e-6853-43e9-9749-c413ec7404e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:50:36.174286 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87be535e-6853-43e9-9749-c413ec7404e7 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:50:36.177978 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eea9b04-6f0d-483c-b933-4bace4691a22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76817cd541294331b0023069092e8e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:50:36.178594 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eea9b04-6f0d-483c-b933-4bace4691a22 service neutron] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:50:36.178874 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eea9b04-6f0d-483c-b933-4bace4691a22 service neutron] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:50:36.179148 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eea9b04-6f0d-483c-b933-4bace4691a22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:50:36.189759 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eea9b04-6f0d-483c-b933-4bace4691a22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:50:36.191199 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eea9b04-6f0d-483c-b933-4bace4691a22 service neutron] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:50:36.200975 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1218/2460] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:50:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:50:40.221839 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b047584-1736-4204-af41-52662493a943 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:50:40.222365 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b047584-1736-4204-af41-52662493a943 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:50:40.222544 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b047584-1736-4204-af41-52662493a943 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:50:40.247712 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-9b047584-1736-4204-af41-52662493a943 None None] MFA Rules not processed for user `c44387d13a694ee6bb47a13906003330`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:50:40.249043 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-9b047584-1736-4204-af41-52662493a943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:50:40.289532 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:50:40.330978 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1243/2461] 10.222.0.89 () {58 vars in 967 bytes} [Thu Jul 28 17:50:40 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:50:40.420961 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eea9b04-6f0d-483c-b933-4bace4691a22 service neutron] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:50:40.424370 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-8eea9b04-6f0d-483c-b933-4bace4691a22 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:50:40.547245 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eea9b04-6f0d-483c-b933-4bace4691a22 service neutron] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:50:40.553692 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71596f6-fed0-4fcf-92fd-38f9d98d8777 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c59473b01bf4433ba3b111114fc048b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:50:40.554634 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71596f6-fed0-4fcf-92fd-38f9d98d8777 service swift] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:50:40.555240 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71596f6-fed0-4fcf-92fd-38f9d98d8777 service swift] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:50:40.555701 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71596f6-fed0-4fcf-92fd-38f9d98d8777 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:50:40.566937 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71596f6-fed0-4fcf-92fd-38f9d98d8777 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:50:40.572650 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71596f6-fed0-4fcf-92fd-38f9d98d8777 service swift] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:50:40.585951 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1219/2462] 10.222.0.89 () {60 vars in 1384 bytes} [Thu Jul 28 17:50:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 17:50:41.299497 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b3ea87-4282-4e76-99b0-8f7b696f237a None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:50:41.300337 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b3ea87-4282-4e76-99b0-8f7b696f237a None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:50:41.300781 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b3ea87-4282-4e76-99b0-8f7b696f237a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:50:41.337868 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-f2b3ea87-4282-4e76-99b0-8f7b696f237a None None] MFA Rules not processed for user `39ff02cba2fd4e7b9fb27b98b9daa82e`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:50:41.338768 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-f2b3ea87-4282-4e76-99b0-8f7b696f237a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:50:41.444928 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1244/2463] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:50:41 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:50:41.460279 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71596f6-fed0-4fcf-92fd-38f9d98d8777 service swift] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:50:41.461911 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-b71596f6-fed0-4fcf-92fd-38f9d98d8777 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:50:41.506615 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:50:41.557552 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71596f6-fed0-4fcf-92fd-38f9d98d8777 service swift] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:50:41.564202 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed0ae50-f334-4ced-8120-ebe5352b991b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd10b49b29df94404a71f43273839c252', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:50:41.564842 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed0ae50-f334-4ced-8120-ebe5352b991b service nova] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:50:41.565121 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed0ae50-f334-4ced-8120-ebe5352b991b service nova] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:50:41.565601 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed0ae50-f334-4ced-8120-ebe5352b991b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:50:41.578583 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed0ae50-f334-4ced-8120-ebe5352b991b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:50:41.580546 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed0ae50-f334-4ced-8120-ebe5352b991b service nova] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:50:41.591140 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1220/2464] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:50:41 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:50:43.289474 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c61df9-0177-4b6e-a82d-f4f117863baf None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:50:43.289995 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c61df9-0177-4b6e-a82d-f4f117863baf None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:50:43.290098 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c61df9-0177-4b6e-a82d-f4f117863baf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:50:43.309314 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-f2c61df9-0177-4b6e-a82d-f4f117863baf None None] MFA Rules not processed for user `39ff02cba2fd4e7b9fb27b98b9daa82e`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:50:43.310545 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-f2c61df9-0177-4b6e-a82d-f4f117863baf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:50:43.317945 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1245/2465] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:50:43 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:50:51.506555 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed0ae50-f334-4ced-8120-ebe5352b991b service nova] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:50:51.508053 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-bed0ae50-f334-4ced-8120-ebe5352b991b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:50:51.580007 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed0ae50-f334-4ced-8120-ebe5352b991b service nova] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:50:51.586105 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8b513b-e1d7-4a42-a263-4929c50b1bcc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4bd290018394b0d955cd376ce35b77f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:50:51.586259 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8b513b-e1d7-4a42-a263-4929c50b1bcc service placement] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:50:51.586522 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8b513b-e1d7-4a42-a263-4929c50b1bcc service placement] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:50:51.586793 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8b513b-e1d7-4a42-a263-4929c50b1bcc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:50:51.590518 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-1a8b513b-e1d7-4a42-a263-4929c50b1bcc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:50:51.605122 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8b513b-e1d7-4a42-a263-4929c50b1bcc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:50:51.606791 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8b513b-e1d7-4a42-a263-4929c50b1bcc service placement] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:50:51.612636 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c61df9-0177-4b6e-a82d-f4f117863baf None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:50:51.614502 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-f2c61df9-0177-4b6e-a82d-f4f117863baf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:50:51.617893 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1221/2466] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:50:51 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 17:50:51.638901 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c61df9-0177-4b6e-a82d-f4f117863baf None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:50:51.645839 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850babe6-c549-4fcc-84c3-d268ff4e7b1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:50:51.646608 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850babe6-c549-4fcc-84c3-d268ff4e7b1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:50:51.647016 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850babe6-c549-4fcc-84c3-d268ff4e7b1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:50:51.647388 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850babe6-c549-4fcc-84c3-d268ff4e7b1c admin admin] PATH_INFO: `/v3/users/ccc41bad96884148a0088cc8048a2316` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:50:51.648819 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850babe6-c549-4fcc-84c3-d268ff4e7b1c admin admin] RBAC: Authorizing `identity:delete_user(user_id=ccc41bad96884148a0088cc8048a2316)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:50:51.650990 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850babe6-c549-4fcc-84c3-d268ff4e7b1c admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:50:51.708213 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-850babe6-c549-4fcc-84c3-d268ff4e7b1c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ccc41bad96884148a0088cc8048a2316'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:50:51.715189 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-850babe6-c549-4fcc-84c3-d268ff4e7b1c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ccc41bad96884148a0088cc8048a2316'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:50:51.715910 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-850babe6-c549-4fcc-84c3-d268ff4e7b1c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ccc41bad96884148a0088cc8048a2316'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:50:51.724633 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-850babe6-c549-4fcc-84c3-d268ff4e7b1c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ccc41bad96884148a0088cc8048a2316'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:50:51.740985 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-850babe6-c549-4fcc-84c3-d268ff4e7b1c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ccc41bad96884148a0088cc8048a2316'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:50:51.747703 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1246/2467] 10.222.0.89 () {58 vars in 1224 bytes} [Thu Jul 28 17:50:51 2022] DELETE /identity/v3/users/ccc41bad96884148a0088cc8048a2316 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:50:52.022861 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8b513b-e1d7-4a42-a263-4929c50b1bcc service placement] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:50:52.035545 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8b513b-e1d7-4a42-a263-4929c50b1bcc service placement] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:50:52.101517 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada7da53-a503-427d-85bb-b516a02e7946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:50:52.102170 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada7da53-a503-427d-85bb-b516a02e7946 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:50:52.102454 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada7da53-a503-427d-85bb-b516a02e7946 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:50:52.102756 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada7da53-a503-427d-85bb-b516a02e7946 admin admin] PATH_INFO: `/v3/projects/0513cb18c0cd43d5a4784f573bb382d4` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:50:52.103958 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada7da53-a503-427d-85bb-b516a02e7946 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0513cb18c0cd43d5a4784f573bb382d4)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:50:52.105948 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada7da53-a503-427d-85bb-b516a02e7946 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:50:52.156398 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:50:52.160354 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-ada7da53-a503-427d-85bb-b516a02e7946 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0513cb18c0cd43d5a4784f573bb382d4', 'request_id': 'req-ada7da53-a503-427d-85bb-b516a02e7946'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:50:52.167061 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-ada7da53-a503-427d-85bb-b516a02e7946 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0513cb18c0cd43d5a4784f573bb382d4', 'request_id': 'req-ada7da53-a503-427d-85bb-b516a02e7946'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:50:52.173179 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-ada7da53-a503-427d-85bb-b516a02e7946 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0513cb18c0cd43d5a4784f573bb382d4', 'request_id': 'req-ada7da53-a503-427d-85bb-b516a02e7946'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:50:52.185791 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-ada7da53-a503-427d-85bb-b516a02e7946 admin admin] The token cache is being invalidate because project 0513cb18c0cd43d5a4784f573bb382d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80325) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 17:50:52.186080 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-ada7da53-a503-427d-85bb-b516a02e7946 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:50:52.187965 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-ada7da53-a503-427d-85bb-b516a02e7946 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:50:52.189624 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1222/2468] 10.222.0.89 () {58 vars in 1233 bytes} [Thu Jul 28 17:50:52 2022] DELETE /identity/v3/projects/0513cb18c0cd43d5a4784f573bb382d4 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:50:52.193593 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850babe6-c549-4fcc-84c3-d268ff4e7b1c admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:50:52.195065 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-850babe6-c549-4fcc-84c3-d268ff4e7b1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:50:52.208676 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850babe6-c549-4fcc-84c3-d268ff4e7b1c admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:50:52.235094 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:50:52.267351 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7e74ba-1742-44aa-8e9d-427960e7d827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:50:52.267990 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7e74ba-1742-44aa-8e9d-427960e7d827 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:50:52.268273 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7e74ba-1742-44aa-8e9d-427960e7d827 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:50:52.268550 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7e74ba-1742-44aa-8e9d-427960e7d827 admin admin] PATH_INFO: `/v3/users/1206b722a36f4c66ad47f762a3e89420` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:50:52.269753 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7e74ba-1742-44aa-8e9d-427960e7d827 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1206b722a36f4c66ad47f762a3e89420)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:50:52.271967 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7e74ba-1742-44aa-8e9d-427960e7d827 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:50:52.320216 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-ad7e74ba-1742-44aa-8e9d-427960e7d827 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1206b722a36f4c66ad47f762a3e89420'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:50:52.327129 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-ad7e74ba-1742-44aa-8e9d-427960e7d827 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1206b722a36f4c66ad47f762a3e89420'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:50:52.327816 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-ad7e74ba-1742-44aa-8e9d-427960e7d827 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1206b722a36f4c66ad47f762a3e89420'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:50:52.335047 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-ad7e74ba-1742-44aa-8e9d-427960e7d827 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1206b722a36f4c66ad47f762a3e89420'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:50:52.347113 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-ad7e74ba-1742-44aa-8e9d-427960e7d827 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1206b722a36f4c66ad47f762a3e89420'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:50:52.352951 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1247/2469] 10.222.0.89 () {58 vars in 1224 bytes} [Thu Jul 28 17:50:52 2022] DELETE /identity/v3/users/1206b722a36f4c66ad47f762a3e89420 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:50:52.534169 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada7da53-a503-427d-85bb-b516a02e7946 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:50:52.535711 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-ada7da53-a503-427d-85bb-b516a02e7946 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:50:52.550114 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada7da53-a503-427d-85bb-b516a02e7946 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:50:52.610950 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c4b348-2b59-4a8e-be44-dc8ff1411692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:50:52.611617 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c4b348-2b59-4a8e-be44-dc8ff1411692 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:50:52.611900 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c4b348-2b59-4a8e-be44-dc8ff1411692 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:50:52.612171 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c4b348-2b59-4a8e-be44-dc8ff1411692 admin admin] PATH_INFO: `/v3/projects/85eb8e598c144cf5a55a4ec5d99a027c` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:50:52.613393 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c4b348-2b59-4a8e-be44-dc8ff1411692 admin admin] RBAC: Authorizing `identity:delete_project(project_id=85eb8e598c144cf5a55a4ec5d99a027c)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:50:52.615330 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c4b348-2b59-4a8e-be44-dc8ff1411692 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:50:52.668671 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-51c4b348-2b59-4a8e-be44-dc8ff1411692 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85eb8e598c144cf5a55a4ec5d99a027c', 'request_id': 'req-51c4b348-2b59-4a8e-be44-dc8ff1411692'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:50:52.675747 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-51c4b348-2b59-4a8e-be44-dc8ff1411692 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85eb8e598c144cf5a55a4ec5d99a027c', 'request_id': 'req-51c4b348-2b59-4a8e-be44-dc8ff1411692'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:50:52.681299 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-51c4b348-2b59-4a8e-be44-dc8ff1411692 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85eb8e598c144cf5a55a4ec5d99a027c', 'request_id': 'req-51c4b348-2b59-4a8e-be44-dc8ff1411692'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:50:52.693910 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-51c4b348-2b59-4a8e-be44-dc8ff1411692 admin admin] The token cache is being invalidate because project 85eb8e598c144cf5a55a4ec5d99a027c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80325) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 17:50:52.694202 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-51c4b348-2b59-4a8e-be44-dc8ff1411692 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:50:52.694825 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-51c4b348-2b59-4a8e-be44-dc8ff1411692 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:50:52.696410 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1223/2470] 10.222.0.89 () {58 vars in 1233 bytes} [Thu Jul 28 17:50:52 2022] DELETE /identity/v3/projects/85eb8e598c144cf5a55a4ec5d99a027c => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:51:02.349553 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=40,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:51:02.690868 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=59,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:51:36.900927 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b64682-11a3-45c0-aa6f-05d96957c12a None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:51:36.901450 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b64682-11a3-45c0-aa6f-05d96957c12a None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:51:36.901554 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b64682-11a3-45c0-aa6f-05d96957c12a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:51:36.920937 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-95b64682-11a3-45c0-aa6f-05d96957c12a None None] MFA Rules not processed for user `c44387d13a694ee6bb47a13906003330`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:51:36.922190 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-95b64682-11a3-45c0-aa6f-05d96957c12a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:51:36.980307 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1248/2471] 10.222.0.89 () {58 vars in 1009 bytes} [Thu Jul 28 17:51:36 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:51:36.993684 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c4b348-2b59-4a8e-be44-dc8ff1411692 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:51:36.995088 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-51c4b348-2b59-4a8e-be44-dc8ff1411692 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:51:37.068175 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c4b348-2b59-4a8e-be44-dc8ff1411692 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:51:37.073237 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e464101-947e-499b-9ff7-f12a99c83997 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c59473b01bf4433ba3b111114fc048b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:51:37.073857 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e464101-947e-499b-9ff7-f12a99c83997 service swift] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:51:37.074139 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e464101-947e-499b-9ff7-f12a99c83997 service swift] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:51:37.074415 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e464101-947e-499b-9ff7-f12a99c83997 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:51:37.083014 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e464101-947e-499b-9ff7-f12a99c83997 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:51:37.084591 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e464101-947e-499b-9ff7-f12a99c83997 service swift] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:51:37.092662 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1224/2472] 10.222.0.89 () {60 vars in 1384 bytes} [Thu Jul 28 17:51:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 17:51:45.217861 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814c8c26-7f3c-428a-99f1-b0946eeb2383 None None] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:51:45.218348 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814c8c26-7f3c-428a-99f1-b0946eeb2383 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:51:45.218456 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814c8c26-7f3c-428a-99f1-b0946eeb2383 None None] PATH_INFO: `/` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:51:45.220062 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1249/2473] 10.222.0.89 () {54 vars in 921 bytes} [Thu Jul 28 17:51:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:51:45.232098 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02de72be-6e9e-4d3f-81cc-b19c9e93262b None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:51:45.232652 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02de72be-6e9e-4d3f-81cc-b19c9e93262b None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:51:45.232652 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02de72be-6e9e-4d3f-81cc-b19c9e93262b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:51:45.265914 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-02de72be-6e9e-4d3f-81cc-b19c9e93262b None None] MFA Rules not processed for user `76817cd541294331b0023069092e8e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:51:45.267763 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-02de72be-6e9e-4d3f-81cc-b19c9e93262b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:51:45.365367 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1225/2474] 10.222.0.89 () {58 vars in 1018 bytes} [Thu Jul 28 17:51:45 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 17:51:45.367264 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814c8c26-7f3c-428a-99f1-b0946eeb2383 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:51:45.369540 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-814c8c26-7f3c-428a-99f1-b0946eeb2383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:51:45.387915 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814c8c26-7f3c-428a-99f1-b0946eeb2383 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:51:45.391929 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90429974-3221-491e-972a-9e8dc8c5edb8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76817cd541294331b0023069092e8e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:51:45.392552 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90429974-3221-491e-972a-9e8dc8c5edb8 service neutron] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:51:45.392834 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90429974-3221-491e-972a-9e8dc8c5edb8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:51:45.393106 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90429974-3221-491e-972a-9e8dc8c5edb8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:51:45.402993 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90429974-3221-491e-972a-9e8dc8c5edb8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:51:45.404441 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90429974-3221-491e-972a-9e8dc8c5edb8 service neutron] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:51:45.425948 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1250/2475] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:51:45 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:51:45.526304 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aae9d2-5add-4588-b974-321aafa20b33 None None] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:51:45.526784 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aae9d2-5add-4588-b974-321aafa20b33 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:51:45.527389 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aae9d2-5add-4588-b974-321aafa20b33 None None] PATH_INFO: `/` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:51:45.528880 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1226/2476] 10.222.0.89 () {54 vars in 921 bytes} [Thu Jul 28 17:51:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 17:51:45.533929 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b422c3-cfb3-46c9-aa1e-730126d0899c None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:51:45.534298 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b422c3-cfb3-46c9-aa1e-730126d0899c None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:51:45.535390 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b422c3-cfb3-46c9-aa1e-730126d0899c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:51:45.561636 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-97b422c3-cfb3-46c9-aa1e-730126d0899c None None] MFA Rules not processed for user `d10b49b29df94404a71f43273839c252`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:51:45.563194 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-97b422c3-cfb3-46c9-aa1e-730126d0899c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:51:45.649915 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1251/2477] 10.222.0.89 () {58 vars in 1018 bytes} [Thu Jul 28 17:51:45 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:51:45.662286 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37aae9d2-5add-4588-b974-321aafa20b33 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:51:45.676660 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37aae9d2-5add-4588-b974-321aafa20b33 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:51:45.680643 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3d6650-8b72-49d7-a2ae-a1c97c124222 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76817cd541294331b0023069092e8e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:51:45.681349 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3d6650-8b72-49d7-a2ae-a1c97c124222 service neutron] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:51:45.681634 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3d6650-8b72-49d7-a2ae-a1c97c124222 service neutron] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:51:45.681911 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3d6650-8b72-49d7-a2ae-a1c97c124222 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:51:45.694929 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3d6650-8b72-49d7-a2ae-a1c97c124222 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:51:45.696380 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3d6650-8b72-49d7-a2ae-a1c97c124222 service neutron] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:51:45.710045 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1227/2478] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:51:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:51:53.727635 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78804e6f-8ddc-4bea-895e-cc455c92f19f None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:51:53.728111 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78804e6f-8ddc-4bea-895e-cc455c92f19f None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:51:53.728208 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78804e6f-8ddc-4bea-895e-cc455c92f19f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:51:53.746954 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-78804e6f-8ddc-4bea-895e-cc455c92f19f None None] MFA Rules not processed for user `c44387d13a694ee6bb47a13906003330`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:51:53.748169 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-78804e6f-8ddc-4bea-895e-cc455c92f19f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:51:53.755388 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1252/2479] 10.222.0.89 () {58 vars in 1009 bytes} [Thu Jul 28 17:51:53 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:51:53.767299 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3d6650-8b72-49d7-a2ae-a1c97c124222 service neutron] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:51:53.779413 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3d6650-8b72-49d7-a2ae-a1c97c124222 service neutron] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:51:53.784418 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dc49df-a73c-4c23-9225-faa659dd293c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c59473b01bf4433ba3b111114fc048b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:51:53.785024 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dc49df-a73c-4c23-9225-faa659dd293c service swift] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:51:53.785340 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dc49df-a73c-4c23-9225-faa659dd293c service swift] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:51:53.785618 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dc49df-a73c-4c23-9225-faa659dd293c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:51:53.794684 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dc49df-a73c-4c23-9225-faa659dd293c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:51:53.796259 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dc49df-a73c-4c23-9225-faa659dd293c service swift] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:51:53.804494 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1228/2480] 10.222.0.89 () {60 vars in 1384 bytes} [Thu Jul 28 17:51:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 17:52:03.741722 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=65 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:03.801128 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=67 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:06.484234 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8736acc-b221-4c5d-ae3f-02be6d4acb86 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:06.484770 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8736acc-b221-4c5d-ae3f-02be6d4acb86 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:06.484770 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8736acc-b221-4c5d-ae3f-02be6d4acb86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:06.504660 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-f8736acc-b221-4c5d-ae3f-02be6d4acb86 None None] MFA Rules not processed for user `39ff02cba2fd4e7b9fb27b98b9daa82e`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:52:06.506159 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-f8736acc-b221-4c5d-ae3f-02be6d4acb86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:52:06.562889 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1253/2481] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:52:06 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:52:09.380548 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5bde63-a5e8-4df2-b25c-1f615bf4888d None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:09.381699 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5bde63-a5e8-4df2-b25c-1f615bf4888d None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:09.381934 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5bde63-a5e8-4df2-b25c-1f615bf4888d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:09.407930 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-3d5bde63-a5e8-4df2-b25c-1f615bf4888d None None] MFA Rules not processed for user `c44387d13a694ee6bb47a13906003330`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:52:09.409189 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-3d5bde63-a5e8-4df2-b25c-1f615bf4888d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:52:09.424085 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1229/2482] 10.222.0.89 () {58 vars in 967 bytes} [Thu Jul 28 17:52:09 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:52:09.432523 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8736acc-b221-4c5d-ae3f-02be6d4acb86 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:09.446595 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8736acc-b221-4c5d-ae3f-02be6d4acb86 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:09.458317 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d94127-2253-4fbb-9d9e-e01de4ff398f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c59473b01bf4433ba3b111114fc048b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:09.459072 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d94127-2253-4fbb-9d9e-e01de4ff398f service swift] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:09.459356 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d94127-2253-4fbb-9d9e-e01de4ff398f service swift] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:09.459771 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d94127-2253-4fbb-9d9e-e01de4ff398f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:09.468850 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d94127-2253-4fbb-9d9e-e01de4ff398f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:09.470485 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d94127-2253-4fbb-9d9e-e01de4ff398f service swift] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:09.480180 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1254/2483] 10.222.0.89 () {60 vars in 1384 bytes} [Thu Jul 28 17:52:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 17:52:19.399772 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=3 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:19.476052 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=31 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:21.471886 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5bde63-a5e8-4df2-b25c-1f615bf4888d None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:21.473398 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-3d5bde63-a5e8-4df2-b25c-1f615bf4888d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:52:21.485839 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5bde63-a5e8-4df2-b25c-1f615bf4888d None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:21.489511 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8802aa6-8ab4-45ec-ad51-9d03de0d3962 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76817cd541294331b0023069092e8e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:21.490146 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8802aa6-8ab4-45ec-ad51-9d03de0d3962 service neutron] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:21.490424 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8802aa6-8ab4-45ec-ad51-9d03de0d3962 service neutron] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:21.491531 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8802aa6-8ab4-45ec-ad51-9d03de0d3962 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:21.493590 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-d8802aa6-8ab4-45ec-ad51-9d03de0d3962 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:52:21.566203 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8802aa6-8ab4-45ec-ad51-9d03de0d3962 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:21.567655 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8802aa6-8ab4-45ec-ad51-9d03de0d3962 service neutron] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:21.579603 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1230/2484] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:52:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 17:52:25.027794 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d94127-2253-4fbb-9d9e-e01de4ff398f service swift] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:25.029401 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-04d94127-2253-4fbb-9d9e-e01de4ff398f service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:52:25.116242 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8802aa6-8ab4-45ec-ad51-9d03de0d3962 service neutron] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:25.118601 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-d8802aa6-8ab4-45ec-ad51-9d03de0d3962 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:52:25.203927 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d94127-2253-4fbb-9d9e-e01de4ff398f service swift] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:25.213172 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b518fc-e229-49fa-bb03-59dafd5a0378 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4bd290018394b0d955cd376ce35b77f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:25.214315 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b518fc-e229-49fa-bb03-59dafd5a0378 service placement] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:25.214796 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b518fc-e229-49fa-bb03-59dafd5a0378 service placement] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:25.215221 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b518fc-e229-49fa-bb03-59dafd5a0378 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:25.217745 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-d9b518fc-e229-49fa-bb03-59dafd5a0378 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:52:25.219341 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8802aa6-8ab4-45ec-ad51-9d03de0d3962 service neutron] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:25.230198 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcc3d7c-b1cc-4d0e-84aa-794106aa9b83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:25.231246 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcc3d7c-b1cc-4d0e-84aa-794106aa9b83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:25.231705 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcc3d7c-b1cc-4d0e-84aa-794106aa9b83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:25.232161 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcc3d7c-b1cc-4d0e-84aa-794106aa9b83 admin admin] PATH_INFO: `/v3/users/22479a66b0f34038b144f4def1aa3f43` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:25.234045 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcc3d7c-b1cc-4d0e-84aa-794106aa9b83 admin admin] RBAC: Authorizing `identity:delete_user(user_id=22479a66b0f34038b144f4def1aa3f43)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:25.237267 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcc3d7c-b1cc-4d0e-84aa-794106aa9b83 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:25.238225 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b518fc-e229-49fa-bb03-59dafd5a0378 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:25.240963 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b518fc-e229-49fa-bb03-59dafd5a0378 service placement] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:25.255337 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1255/2485] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:52:25 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:52:25.302214 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-efcc3d7c-b1cc-4d0e-84aa-794106aa9b83 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22479a66b0f34038b144f4def1aa3f43'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:52:25.309065 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-efcc3d7c-b1cc-4d0e-84aa-794106aa9b83 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22479a66b0f34038b144f4def1aa3f43'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:52:25.317344 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-efcc3d7c-b1cc-4d0e-84aa-794106aa9b83 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22479a66b0f34038b144f4def1aa3f43'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:52:25.318038 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-efcc3d7c-b1cc-4d0e-84aa-794106aa9b83 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22479a66b0f34038b144f4def1aa3f43'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:52:25.318710 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-efcc3d7c-b1cc-4d0e-84aa-794106aa9b83 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22479a66b0f34038b144f4def1aa3f43'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:52:25.340953 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1231/2486] 10.222.0.89 () {58 vars in 1224 bytes} [Thu Jul 28 17:52:25 2022] DELETE /identity/v3/users/22479a66b0f34038b144f4def1aa3f43 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:52:25.689901 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b518fc-e229-49fa-bb03-59dafd5a0378 service placement] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:25.701192 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b518fc-e229-49fa-bb03-59dafd5a0378 service placement] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:25.792693 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b941af1-841b-41fd-98ca-fd15bcc502bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:25.793544 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b941af1-841b-41fd-98ca-fd15bcc502bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:25.794042 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b941af1-841b-41fd-98ca-fd15bcc502bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:25.794510 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b941af1-841b-41fd-98ca-fd15bcc502bc admin admin] PATH_INFO: `/v3/projects/78a2f25af7784086aadec8dd5a1015dd` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:25.795806 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b941af1-841b-41fd-98ca-fd15bcc502bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=78a2f25af7784086aadec8dd5a1015dd)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:25.798486 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b941af1-841b-41fd-98ca-fd15bcc502bc admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:25.852414 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:25.857334 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-7b941af1-841b-41fd-98ca-fd15bcc502bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78a2f25af7784086aadec8dd5a1015dd', 'request_id': 'req-7b941af1-841b-41fd-98ca-fd15bcc502bc'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:52:25.866350 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-7b941af1-841b-41fd-98ca-fd15bcc502bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78a2f25af7784086aadec8dd5a1015dd', 'request_id': 'req-7b941af1-841b-41fd-98ca-fd15bcc502bc'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:52:25.874474 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-7b941af1-841b-41fd-98ca-fd15bcc502bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78a2f25af7784086aadec8dd5a1015dd', 'request_id': 'req-7b941af1-841b-41fd-98ca-fd15bcc502bc'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:52:25.894017 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-7b941af1-841b-41fd-98ca-fd15bcc502bc admin admin] The token cache is being invalidate because project 78a2f25af7784086aadec8dd5a1015dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80324) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 17:52:25.894471 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-7b941af1-841b-41fd-98ca-fd15bcc502bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:52:25.895317 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-7b941af1-841b-41fd-98ca-fd15bcc502bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:52:25.897283 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1256/2487] 10.222.0.89 () {58 vars in 1233 bytes} [Thu Jul 28 17:52:25 2022] DELETE /identity/v3/projects/78a2f25af7784086aadec8dd5a1015dd => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:52:25.902435 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ec002c-e1be-421c-ad2b-adafadbcf3f9 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:25.902729 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ec002c-e1be-421c-ad2b-adafadbcf3f9 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:25.903062 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ec002c-e1be-421c-ad2b-adafadbcf3f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:25.909324 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:25.978162 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-d0ec002c-e1be-421c-ad2b-adafadbcf3f9 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:52:25.980401 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-d0ec002c-e1be-421c-ad2b-adafadbcf3f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:52:26.055696 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1232/2488] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:52:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 17:52:26.066824 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a1099c-40a3-4ab1-8285-f3ebb84b0120 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:26.067941 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a1099c-40a3-4ab1-8285-f3ebb84b0120 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:26.068389 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a1099c-40a3-4ab1-8285-f3ebb84b0120 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:26.093522 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-c9a1099c-40a3-4ab1-8285-f3ebb84b0120 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:52:26.097421 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-c9a1099c-40a3-4ab1-8285-f3ebb84b0120 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:52:26.110126 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1257/2489] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:52:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:52:26.114176 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ec002c-e1be-421c-ad2b-adafadbcf3f9 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:26.127853 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ec002c-e1be-421c-ad2b-adafadbcf3f9 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:26.135614 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7f65a2-e57a-4256-9982-e279dc66e406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:26.136775 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7f65a2-e57a-4256-9982-e279dc66e406 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:26.137268 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7f65a2-e57a-4256-9982-e279dc66e406 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:26.137606 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7f65a2-e57a-4256-9982-e279dc66e406 admin admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:26.138330 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7f65a2-e57a-4256-9982-e279dc66e406 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:52:26.138641 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7f65a2-e57a-4256-9982-e279dc66e406 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:26.140311 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7f65a2-e57a-4256-9982-e279dc66e406 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:26.155974 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1233/2490] 10.222.0.89 () {58 vars in 1153 bytes} [Thu Jul 28 17:52:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:52:26.160177 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a1099c-40a3-4ab1-8285-f3ebb84b0120 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:26.170285 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a1099c-40a3-4ab1-8285-f3ebb84b0120 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:26.198017 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43961cc9-84de-4fe0-8e3c-82886472e1d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:26.198575 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43961cc9-84de-4fe0-8e3c-82886472e1d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:26.198858 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43961cc9-84de-4fe0-8e3c-82886472e1d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:26.199132 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43961cc9-84de-4fe0-8e3c-82886472e1d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:26.199827 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43961cc9-84de-4fe0-8e3c-82886472e1d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:26.201779 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43961cc9-84de-4fe0-8e3c-82886472e1d2 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:26.248087 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1258/2491] 10.222.0.89 () {60 vars in 1153 bytes} [Thu Jul 28 17:52:26 2022] POST /identity/v3/projects => generated 373 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:52:26.252246 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7f65a2-e57a-4256-9982-e279dc66e406 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:26.263086 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7f65a2-e57a-4256-9982-e279dc66e406 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:26.341958 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa04aae-e14a-4808-ba08-2b4a487579e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:26.341958 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa04aae-e14a-4808-ba08-2b4a487579e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:26.342292 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa04aae-e14a-4808-ba08-2b4a487579e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:26.342534 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa04aae-e14a-4808-ba08-2b4a487579e8 admin admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:26.343211 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa04aae-e14a-4808-ba08-2b4a487579e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:26.345448 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa04aae-e14a-4808-ba08-2b4a487579e8 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:26.371866 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1234/2492] 10.222.0.89 () {60 vars in 1144 bytes} [Thu Jul 28 17:52:26 2022] POST /identity/v3/users => generated 287 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 17:52:26.378266 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43961cc9-84de-4fe0-8e3c-82886472e1d2 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:26.392153 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43961cc9-84de-4fe0-8e3c-82886472e1d2 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:26.399737 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dda9a29-f785-475c-94cc-985e1f03de9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:26.400454 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dda9a29-f785-475c-94cc-985e1f03de9e admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:26.400800 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dda9a29-f785-475c-94cc-985e1f03de9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:26.401199 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dda9a29-f785-475c-94cc-985e1f03de9e admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:26.401969 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dda9a29-f785-475c-94cc-985e1f03de9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:26.404144 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dda9a29-f785-475c-94cc-985e1f03de9e admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:26.418431 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1259/2493] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:52:26 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 17:52:26.424554 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa04aae-e14a-4808-ba08-2b4a487579e8 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:26.436105 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa04aae-e14a-4808-ba08-2b4a487579e8 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:26.441270 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376b5de2-9804-4f6f-9143-6faa88b6cd8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:26.442015 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376b5de2-9804-4f6f-9143-6faa88b6cd8b admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:26.442538 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376b5de2-9804-4f6f-9143-6faa88b6cd8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:26.442926 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376b5de2-9804-4f6f-9143-6faa88b6cd8b admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:26.444307 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376b5de2-9804-4f6f-9143-6faa88b6cd8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:26.446269 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376b5de2-9804-4f6f-9143-6faa88b6cd8b admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:26.459105 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1235/2494] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:52:26 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 17:52:26.464881 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dda9a29-f785-475c-94cc-985e1f03de9e admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:26.477032 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dda9a29-f785-475c-94cc-985e1f03de9e admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:26.482631 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e763c0c1-4160-4a85-a0b5-f709406a43ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:26.483315 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e763c0c1-4160-4a85-a0b5-f709406a43ea admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:26.483961 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e763c0c1-4160-4a85-a0b5-f709406a43ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:26.483961 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e763c0c1-4160-4a85-a0b5-f709406a43ea admin admin] PATH_INFO: `/v3/projects/41427b4bd1954e39b7788ceb3beeb6ea/users/281bce48542044b9a781465e7140e9ed/roles/11f8731756fe4caca13efa61b71b11c5` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:26.503189 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e763c0c1-4160-4a85-a0b5-f709406a43ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=41427b4bd1954e39b7788ceb3beeb6ea, user_id=281bce48542044b9a781465e7140e9ed, role_id=11f8731756fe4caca13efa61b71b11c5)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:26.505923 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e763c0c1-4160-4a85-a0b5-f709406a43ea admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:26.529669 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1260/2495] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:52:26 2022] PUT /identity/v3/projects/41427b4bd1954e39b7788ceb3beeb6ea/users/281bce48542044b9a781465e7140e9ed/roles/11f8731756fe4caca13efa61b71b11c5 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:52:26.540000 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafbf3fe-6e91-4388-8dc2-0d8f833454d4 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:26.541264 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafbf3fe-6e91-4388-8dc2-0d8f833454d4 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:26.541755 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafbf3fe-6e91-4388-8dc2-0d8f833454d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:26.582296 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-aafbf3fe-6e91-4388-8dc2-0d8f833454d4 None None] MFA Rules not processed for user `281bce48542044b9a781465e7140e9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:52:26.583615 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-aafbf3fe-6e91-4388-8dc2-0d8f833454d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:52:26.619291 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:26.671700 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1236/2496] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:52:26 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:52:26.684180 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e763c0c1-4160-4a85-a0b5-f709406a43ea admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:26.691504 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-e763c0c1-4160-4a85-a0b5-f709406a43ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:52:26.818618 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e763c0c1-4160-4a85-a0b5-f709406a43ea admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:26.827236 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21085e4d-d98f-412d-b1ea-dd371184f18c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd10b49b29df94404a71f43273839c252', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:26.828141 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21085e4d-d98f-412d-b1ea-dd371184f18c service nova] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:26.828557 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21085e4d-d98f-412d-b1ea-dd371184f18c service nova] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:26.828968 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21085e4d-d98f-412d-b1ea-dd371184f18c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:26.840979 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21085e4d-d98f-412d-b1ea-dd371184f18c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:26.842621 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21085e4d-d98f-412d-b1ea-dd371184f18c service nova] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:26.854025 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1261/2497] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:52:26 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:52:29.549572 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafbf3fe-6e91-4388-8dc2-0d8f833454d4 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:29.612265 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafbf3fe-6e91-4388-8dc2-0d8f833454d4 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:29.616847 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b083ac-29e5-4f73-8f24-e0c07208dce7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:29.617523 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b083ac-29e5-4f73-8f24-e0c07208dce7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:29.617804 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b083ac-29e5-4f73-8f24-e0c07208dce7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:29.618080 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b083ac-29e5-4f73-8f24-e0c07208dce7 admin admin] PATH_INFO: `/v3/users/281bce48542044b9a781465e7140e9ed` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:29.619247 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b083ac-29e5-4f73-8f24-e0c07208dce7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=281bce48542044b9a781465e7140e9ed)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:29.621223 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b083ac-29e5-4f73-8f24-e0c07208dce7 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:29.673578 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-d5b083ac-29e5-4f73-8f24-e0c07208dce7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '281bce48542044b9a781465e7140e9ed'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:52:29.680434 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-d5b083ac-29e5-4f73-8f24-e0c07208dce7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '281bce48542044b9a781465e7140e9ed'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:52:29.687826 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-d5b083ac-29e5-4f73-8f24-e0c07208dce7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '281bce48542044b9a781465e7140e9ed'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:52:29.688600 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-d5b083ac-29e5-4f73-8f24-e0c07208dce7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '281bce48542044b9a781465e7140e9ed'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:52:29.689310 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-d5b083ac-29e5-4f73-8f24-e0c07208dce7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '281bce48542044b9a781465e7140e9ed'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:52:29.706740 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1237/2498] 10.222.0.89 () {58 vars in 1224 bytes} [Thu Jul 28 17:52:29 2022] DELETE /identity/v3/users/281bce48542044b9a781465e7140e9ed => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:52:29.712366 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb9bfa5-d779-42f8-91e2-9c2de990d51e None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:29.712894 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb9bfa5-d779-42f8-91e2-9c2de990d51e None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:29.712894 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb9bfa5-d779-42f8-91e2-9c2de990d51e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:29.732783 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-4cb9bfa5-d779-42f8-91e2-9c2de990d51e None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:52:29.734014 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-4cb9bfa5-d779-42f8-91e2-9c2de990d51e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:52:29.753770 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:29.795213 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1262/2499] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:52:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:52:29.804993 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b083ac-29e5-4f73-8f24-e0c07208dce7 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:29.806424 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-d5b083ac-29e5-4f73-8f24-e0c07208dce7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:52:29.860927 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:29.865207 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b083ac-29e5-4f73-8f24-e0c07208dce7 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:29.870480 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d158af-2f22-48d5-9d1f-f6c35a7eed6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76817cd541294331b0023069092e8e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:29.871308 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d158af-2f22-48d5-9d1f-f6c35a7eed6e service neutron] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:29.871660 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d158af-2f22-48d5-9d1f-f6c35a7eed6e service neutron] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:29.872094 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d158af-2f22-48d5-9d1f-f6c35a7eed6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:29.881845 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d158af-2f22-48d5-9d1f-f6c35a7eed6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:29.883264 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d158af-2f22-48d5-9d1f-f6c35a7eed6e service neutron] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:29.892707 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1238/2500] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:52:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:52:30.165731 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb9bfa5-d779-42f8-91e2-9c2de990d51e None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:30.167255 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-4cb9bfa5-d779-42f8-91e2-9c2de990d51e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:52:30.179927 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb9bfa5-d779-42f8-91e2-9c2de990d51e None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:30.184772 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ec5868-1074-4951-b13a-e810d79bfce9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:30.185515 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ec5868-1074-4951-b13a-e810d79bfce9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:30.185878 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ec5868-1074-4951-b13a-e810d79bfce9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:30.186066 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ec5868-1074-4951-b13a-e810d79bfce9 admin admin] PATH_INFO: `/v3/projects/41427b4bd1954e39b7788ceb3beeb6ea` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:30.187239 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ec5868-1074-4951-b13a-e810d79bfce9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=41427b4bd1954e39b7788ceb3beeb6ea)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:30.189142 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ec5868-1074-4951-b13a-e810d79bfce9 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:30.237405 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-09ec5868-1074-4951-b13a-e810d79bfce9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41427b4bd1954e39b7788ceb3beeb6ea', 'request_id': 'req-09ec5868-1074-4951-b13a-e810d79bfce9'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:52:30.243568 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-09ec5868-1074-4951-b13a-e810d79bfce9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41427b4bd1954e39b7788ceb3beeb6ea', 'request_id': 'req-09ec5868-1074-4951-b13a-e810d79bfce9'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:52:30.249714 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-09ec5868-1074-4951-b13a-e810d79bfce9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41427b4bd1954e39b7788ceb3beeb6ea', 'request_id': 'req-09ec5868-1074-4951-b13a-e810d79bfce9'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:52:30.263102 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-09ec5868-1074-4951-b13a-e810d79bfce9 admin admin] The token cache is being invalidate because project 41427b4bd1954e39b7788ceb3beeb6ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80324) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 17:52:30.263383 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-09ec5868-1074-4951-b13a-e810d79bfce9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:52:30.264027 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-09ec5868-1074-4951-b13a-e810d79bfce9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:52:30.265567 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1263/2501] 10.222.0.89 () {58 vars in 1233 bytes} [Thu Jul 28 17:52:30 2022] DELETE /identity/v3/projects/41427b4bd1954e39b7788ceb3beeb6ea => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:52:30.270779 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f954a585-18fc-4316-9cac-d7c64d44e394 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:30.271074 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f954a585-18fc-4316-9cac-d7c64d44e394 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:30.271351 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f954a585-18fc-4316-9cac-d7c64d44e394 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:30.291528 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-f954a585-18fc-4316-9cac-d7c64d44e394 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:52:30.292756 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-f954a585-18fc-4316-9cac-d7c64d44e394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:52:30.351753 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1239/2502] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:52:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 17:52:30.359545 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8324ea0a-19a8-4a21-9806-0eb0c88a0c56 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:30.359827 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8324ea0a-19a8-4a21-9806-0eb0c88a0c56 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:30.360102 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8324ea0a-19a8-4a21-9806-0eb0c88a0c56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:30.380375 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-8324ea0a-19a8-4a21-9806-0eb0c88a0c56 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:52:30.382224 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-8324ea0a-19a8-4a21-9806-0eb0c88a0c56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:52:30.390228 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1264/2503] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:52:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:52:30.394420 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f954a585-18fc-4316-9cac-d7c64d44e394 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:30.404443 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f954a585-18fc-4316-9cac-d7c64d44e394 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:30.409044 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20d0664-da3a-4f99-9897-33867895e9c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:30.409737 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20d0664-da3a-4f99-9897-33867895e9c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:30.410002 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20d0664-da3a-4f99-9897-33867895e9c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:30.410278 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20d0664-da3a-4f99-9897-33867895e9c3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:30.410976 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20d0664-da3a-4f99-9897-33867895e9c3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:52:30.411280 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20d0664-da3a-4f99-9897-33867895e9c3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:30.412928 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20d0664-da3a-4f99-9897-33867895e9c3 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:30.424318 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1240/2504] 10.222.0.89 () {58 vars in 1153 bytes} [Thu Jul 28 17:52:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:52:30.429369 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8324ea0a-19a8-4a21-9806-0eb0c88a0c56 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:30.438591 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8324ea0a-19a8-4a21-9806-0eb0c88a0c56 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:30.443285 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7540b983-4957-48b0-abd9-1c5bd336ee44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:30.444485 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7540b983-4957-48b0-abd9-1c5bd336ee44 admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:30.444877 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7540b983-4957-48b0-abd9-1c5bd336ee44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:30.445152 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7540b983-4957-48b0-abd9-1c5bd336ee44 admin admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:30.445864 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7540b983-4957-48b0-abd9-1c5bd336ee44 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:30.447759 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7540b983-4957-48b0-abd9-1c5bd336ee44 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:30.480944 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1265/2505] 10.222.0.89 () {60 vars in 1153 bytes} [Thu Jul 28 17:52:30 2022] POST /identity/v3/projects => generated 357 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:52:30.485195 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20d0664-da3a-4f99-9897-33867895e9c3 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:30.495690 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20d0664-da3a-4f99-9897-33867895e9c3 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:30.552313 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9214b092-1f22-4259-9566-fec1e91515d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:30.553031 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9214b092-1f22-4259-9566-fec1e91515d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:30.553839 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9214b092-1f22-4259-9566-fec1e91515d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:30.554118 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9214b092-1f22-4259-9566-fec1e91515d2 admin admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:30.554791 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9214b092-1f22-4259-9566-fec1e91515d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:30.556702 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9214b092-1f22-4259-9566-fec1e91515d2 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:30.579486 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1241/2506] 10.222.0.89 () {60 vars in 1144 bytes} [Thu Jul 28 17:52:30 2022] POST /identity/v3/users => generated 279 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 17:52:30.583716 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7540b983-4957-48b0-abd9-1c5bd336ee44 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:30.594085 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7540b983-4957-48b0-abd9-1c5bd336ee44 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:30.598680 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3291f73-d89a-4aac-95a7-8b0d1ea1b6a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:30.599253 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3291f73-d89a-4aac-95a7-8b0d1ea1b6a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:30.599528 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3291f73-d89a-4aac-95a7-8b0d1ea1b6a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:30.599794 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3291f73-d89a-4aac-95a7-8b0d1ea1b6a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:30.601244 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3291f73-d89a-4aac-95a7-8b0d1ea1b6a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:30.602882 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3291f73-d89a-4aac-95a7-8b0d1ea1b6a4 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:30.612460 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1266/2507] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:52:30 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:52:30.616945 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9214b092-1f22-4259-9566-fec1e91515d2 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:30.627827 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9214b092-1f22-4259-9566-fec1e91515d2 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:30.634433 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3770ef-468e-4f0d-b5f2-cdcbb668bd7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:30.635084 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3770ef-468e-4f0d-b5f2-cdcbb668bd7f admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:30.635371 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3770ef-468e-4f0d-b5f2-cdcbb668bd7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:30.635647 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3770ef-468e-4f0d-b5f2-cdcbb668bd7f admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:30.636348 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3770ef-468e-4f0d-b5f2-cdcbb668bd7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:30.638625 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3770ef-468e-4f0d-b5f2-cdcbb668bd7f admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:30.650936 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1242/2508] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:52:30 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:52:30.655138 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3291f73-d89a-4aac-95a7-8b0d1ea1b6a4 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:30.666286 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3291f73-d89a-4aac-95a7-8b0d1ea1b6a4 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:30.670987 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d522ac21-532b-4f93-951d-10e161bc27ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:30.671640 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d522ac21-532b-4f93-951d-10e161bc27ed admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:30.671923 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d522ac21-532b-4f93-951d-10e161bc27ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:30.672201 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d522ac21-532b-4f93-951d-10e161bc27ed admin admin] PATH_INFO: `/v3/projects/478aae0d800b441f8f16231621a07ad1/users/9e9beaca74ab4950a86e810528472d98/roles/11f8731756fe4caca13efa61b71b11c5` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:30.687691 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d522ac21-532b-4f93-951d-10e161bc27ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=478aae0d800b441f8f16231621a07ad1, user_id=9e9beaca74ab4950a86e810528472d98, role_id=11f8731756fe4caca13efa61b71b11c5)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:30.690471 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d522ac21-532b-4f93-951d-10e161bc27ed admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:30.703797 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1267/2509] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:52:30 2022] PUT /identity/v3/projects/478aae0d800b441f8f16231621a07ad1/users/9e9beaca74ab4950a86e810528472d98/roles/11f8731756fe4caca13efa61b71b11c5 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:52:30.715542 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e385bc-0192-4c23-8942-65dcfdc8d22b None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:30.715828 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e385bc-0192-4c23-8942-65dcfdc8d22b None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:30.716101 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e385bc-0192-4c23-8942-65dcfdc8d22b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:30.744686 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-11e385bc-0192-4c23-8942-65dcfdc8d22b None None] MFA Rules not processed for user `9e9beaca74ab4950a86e810528472d98`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:52:30.745935 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-11e385bc-0192-4c23-8942-65dcfdc8d22b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:52:30.790603 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:30.812857 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1243/2510] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:52:30 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:52:30.817438 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d522ac21-532b-4f93-951d-10e161bc27ed admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:30.867647 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:30.883581 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d522ac21-532b-4f93-951d-10e161bc27ed admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:30.888464 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aef179a-4708-4011-be23-b672da419874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:30.889074 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aef179a-4708-4011-be23-b672da419874 admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:30.889383 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aef179a-4708-4011-be23-b672da419874 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:30.889664 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aef179a-4708-4011-be23-b672da419874 admin admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:30.890324 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aef179a-4708-4011-be23-b672da419874 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:30.892245 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aef179a-4708-4011-be23-b672da419874 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:30.923828 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1268/2511] 10.222.0.89 () {60 vars in 1153 bytes} [Thu Jul 28 17:52:30 2022] POST /identity/v3/projects => generated 357 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:52:30.925398 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e385bc-0192-4c23-8942-65dcfdc8d22b None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:30.936459 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e385bc-0192-4c23-8942-65dcfdc8d22b None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:31.001376 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42682dd8-01ca-475e-a22d-493af9d3f3e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:31.001998 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42682dd8-01ca-475e-a22d-493af9d3f3e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:31.002280 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42682dd8-01ca-475e-a22d-493af9d3f3e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:31.002556 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42682dd8-01ca-475e-a22d-493af9d3f3e9 admin admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:31.003216 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42682dd8-01ca-475e-a22d-493af9d3f3e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:31.005136 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42682dd8-01ca-475e-a22d-493af9d3f3e9 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:31.027593 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1244/2512] 10.222.0.89 () {60 vars in 1144 bytes} [Thu Jul 28 17:52:30 2022] POST /identity/v3/users => generated 285 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:52:31.031867 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aef179a-4708-4011-be23-b672da419874 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:31.042539 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aef179a-4708-4011-be23-b672da419874 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:31.047235 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7282b2-a469-4793-a917-61c908079b3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:31.047843 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7282b2-a469-4793-a917-61c908079b3e admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:31.048123 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7282b2-a469-4793-a917-61c908079b3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:31.048395 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7282b2-a469-4793-a917-61c908079b3e admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:31.049080 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7282b2-a469-4793-a917-61c908079b3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:31.050700 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7282b2-a469-4793-a917-61c908079b3e admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:31.062804 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1269/2513] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:52:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:52:31.067075 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42682dd8-01ca-475e-a22d-493af9d3f3e9 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:31.077866 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42682dd8-01ca-475e-a22d-493af9d3f3e9 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:31.082599 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99820966-bfc1-4816-aa6f-78dd4f5ae4e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:31.083256 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99820966-bfc1-4816-aa6f-78dd4f5ae4e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:31.083539 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99820966-bfc1-4816-aa6f-78dd4f5ae4e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:31.083813 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99820966-bfc1-4816-aa6f-78dd4f5ae4e9 admin admin] PATH_INFO: `/v3/projects/4e1611fe4e2f42fa8eb3010b6bedce2d/users/e1c4c70a288447cbbd1bb9c7f1ca8a38/roles/73e457231a824ea98c28d22610be640c` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:31.227485 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99820966-bfc1-4816-aa6f-78dd4f5ae4e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e1611fe4e2f42fa8eb3010b6bedce2d, user_id=e1c4c70a288447cbbd1bb9c7f1ca8a38, role_id=73e457231a824ea98c28d22610be640c)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:31.230534 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99820966-bfc1-4816-aa6f-78dd4f5ae4e9 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:31.245384 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1245/2514] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:52:31 2022] PUT /identity/v3/projects/4e1611fe4e2f42fa8eb3010b6bedce2d/users/e1c4c70a288447cbbd1bb9c7f1ca8a38/roles/73e457231a824ea98c28d22610be640c => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:52:31.249510 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7282b2-a469-4793-a917-61c908079b3e admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:31.260544 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7282b2-a469-4793-a917-61c908079b3e admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:31.315957 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bcd6c2-4fac-4cef-9c8f-17473c05f4bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:31.316566 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bcd6c2-4fac-4cef-9c8f-17473c05f4bf admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:31.316849 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bcd6c2-4fac-4cef-9c8f-17473c05f4bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:31.317123 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bcd6c2-4fac-4cef-9c8f-17473c05f4bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:31.317834 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bcd6c2-4fac-4cef-9c8f-17473c05f4bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:31.320189 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bcd6c2-4fac-4cef-9c8f-17473c05f4bf admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:31.329762 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1270/2515] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:52:31 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:52:31.334015 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99820966-bfc1-4816-aa6f-78dd4f5ae4e9 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:31.344547 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99820966-bfc1-4816-aa6f-78dd4f5ae4e9 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:31.349185 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7aeef8-504e-47cf-8254-4da6be408b46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:31.349871 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7aeef8-504e-47cf-8254-4da6be408b46 admin admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:31.350794 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7aeef8-504e-47cf-8254-4da6be408b46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:31.351087 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7aeef8-504e-47cf-8254-4da6be408b46 admin admin] PATH_INFO: `/v3/projects/4e1611fe4e2f42fa8eb3010b6bedce2d/users/e1c4c70a288447cbbd1bb9c7f1ca8a38/roles/73e457231a824ea98c28d22610be640c` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:31.353072 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7aeef8-504e-47cf-8254-4da6be408b46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e1611fe4e2f42fa8eb3010b6bedce2d, user_id=e1c4c70a288447cbbd1bb9c7f1ca8a38, role_id=73e457231a824ea98c28d22610be640c)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:31.355744 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7aeef8-504e-47cf-8254-4da6be408b46 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:31.366938 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1246/2516] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:52:31 2022] PUT /identity/v3/projects/4e1611fe4e2f42fa8eb3010b6bedce2d/users/e1c4c70a288447cbbd1bb9c7f1ca8a38/roles/73e457231a824ea98c28d22610be640c => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:52:31.377524 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95d762a-5666-4a8a-9a8d-1cfbad19af15 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:31.377818 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95d762a-5666-4a8a-9a8d-1cfbad19af15 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:31.378091 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95d762a-5666-4a8a-9a8d-1cfbad19af15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:31.405928 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-f95d762a-5666-4a8a-9a8d-1cfbad19af15 None None] MFA Rules not processed for user `e1c4c70a288447cbbd1bb9c7f1ca8a38`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:52:31.407174 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-f95d762a-5666-4a8a-9a8d-1cfbad19af15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:52:31.476217 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1271/2517] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:52:31 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:52:31.487274 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7aeef8-504e-47cf-8254-4da6be408b46 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:31.488956 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-cb7aeef8-504e-47cf-8254-4da6be408b46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:52:31.545454 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7aeef8-504e-47cf-8254-4da6be408b46 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:31.549044 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0693f739-e53e-4812-98ad-0ad4cd986985 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96d512d0c84240918dae6c4a3e8b7b79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:31.549678 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0693f739-e53e-4812-98ad-0ad4cd986985 service cinder] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:31.549954 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0693f739-e53e-4812-98ad-0ad4cd986985 service cinder] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:31.550223 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0693f739-e53e-4812-98ad-0ad4cd986985 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:31.559837 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0693f739-e53e-4812-98ad-0ad4cd986985 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:31.561264 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0693f739-e53e-4812-98ad-0ad4cd986985 service cinder] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:31.570636 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1247/2518] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:52:31 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:52:31.648086 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95d762a-5666-4a8a-9a8d-1cfbad19af15 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:31.657949 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95d762a-5666-4a8a-9a8d-1cfbad19af15 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:31.661578 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2430f1f7-57da-48a9-9a54-20a54f63b08b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96d512d0c84240918dae6c4a3e8b7b79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:31.662176 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2430f1f7-57da-48a9-9a54-20a54f63b08b service cinder] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:31.662449 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2430f1f7-57da-48a9-9a54-20a54f63b08b service cinder] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:31.663776 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2430f1f7-57da-48a9-9a54-20a54f63b08b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:31.704023 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:31.721666 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2430f1f7-57da-48a9-9a54-20a54f63b08b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:31.723183 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2430f1f7-57da-48a9-9a54-20a54f63b08b service cinder] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:31.733470 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1272/2519] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:52:31 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:52:36.786711 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0693f739-e53e-4812-98ad-0ad4cd986985 service cinder] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:36.789211 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-0693f739-e53e-4812-98ad-0ad4cd986985 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:52:36.837658 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:36.883446 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0693f739-e53e-4812-98ad-0ad4cd986985 service cinder] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:36.888430 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bf3190-4663-4371-ac1f-de41ff62bf03 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4bd290018394b0d955cd376ce35b77f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:36.889038 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bf3190-4663-4371-ac1f-de41ff62bf03 service placement] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:36.889347 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bf3190-4663-4371-ac1f-de41ff62bf03 service placement] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:36.889620 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bf3190-4663-4371-ac1f-de41ff62bf03 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:36.891370 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-32bf3190-4663-4371-ac1f-de41ff62bf03 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:52:36.903851 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bf3190-4663-4371-ac1f-de41ff62bf03 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:36.905442 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bf3190-4663-4371-ac1f-de41ff62bf03 service placement] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:36.915269 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1248/2520] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:52:36 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:52:41.728730 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=9 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:46.574220 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2430f1f7-57da-48a9-9a54-20a54f63b08b service cinder] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:46.637970 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2430f1f7-57da-48a9-9a54-20a54f63b08b service cinder] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:46.642507 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4746822c-5e3b-463f-b798-646264e3aa88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:46.643126 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4746822c-5e3b-463f-b798-646264e3aa88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:46.643401 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4746822c-5e3b-463f-b798-646264e3aa88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:46.643699 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4746822c-5e3b-463f-b798-646264e3aa88 admin admin] PATH_INFO: `/v3/users/9e9beaca74ab4950a86e810528472d98` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:46.644866 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4746822c-5e3b-463f-b798-646264e3aa88 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e9beaca74ab4950a86e810528472d98)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:46.647002 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4746822c-5e3b-463f-b798-646264e3aa88 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:46.691534 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-4746822c-5e3b-463f-b798-646264e3aa88 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e9beaca74ab4950a86e810528472d98'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:52:46.698841 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-4746822c-5e3b-463f-b798-646264e3aa88 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e9beaca74ab4950a86e810528472d98'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:52:46.699495 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-4746822c-5e3b-463f-b798-646264e3aa88 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e9beaca74ab4950a86e810528472d98'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:52:46.706687 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-4746822c-5e3b-463f-b798-646264e3aa88 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e9beaca74ab4950a86e810528472d98'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:52:46.719564 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-4746822c-5e3b-463f-b798-646264e3aa88 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e9beaca74ab4950a86e810528472d98'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:52:46.725390 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1273/2521] 10.222.0.89 () {58 vars in 1224 bytes} [Thu Jul 28 17:52:46 2022] DELETE /identity/v3/users/9e9beaca74ab4950a86e810528472d98 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 17:52:46.729820 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca7a726-ead7-4039-b466-53463da22318 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:46.730305 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca7a726-ead7-4039-b466-53463da22318 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:46.730416 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca7a726-ead7-4039-b466-53463da22318 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:46.749146 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-5ca7a726-ead7-4039-b466-53463da22318 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:52:46.750369 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-5ca7a726-ead7-4039-b466-53463da22318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:52:46.808163 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1249/2522] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:52:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:52:46.817945 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4746822c-5e3b-463f-b798-646264e3aa88 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:46.819385 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-4746822c-5e3b-463f-b798-646264e3aa88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:52:46.898033 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4746822c-5e3b-463f-b798-646264e3aa88 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:46.902944 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b5af30-1147-4d1f-adde-c9ce20a7da99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76817cd541294331b0023069092e8e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:46.903558 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b5af30-1147-4d1f-adde-c9ce20a7da99 service neutron] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:46.903839 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b5af30-1147-4d1f-adde-c9ce20a7da99 service neutron] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:46.904112 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b5af30-1147-4d1f-adde-c9ce20a7da99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:46.913800 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b5af30-1147-4d1f-adde-c9ce20a7da99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:46.915241 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b5af30-1147-4d1f-adde-c9ce20a7da99 service neutron] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:46.924905 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1274/2523] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:52:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:52:47.197932 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca7a726-ead7-4039-b466-53463da22318 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:47.199491 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-5ca7a726-ead7-4039-b466-53463da22318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:52:47.216644 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca7a726-ead7-4039-b466-53463da22318 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:47.221358 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1d9aa3-1117-4993-bc20-c961539c8381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:47.222038 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1d9aa3-1117-4993-bc20-c961539c8381 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:47.222318 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1d9aa3-1117-4993-bc20-c961539c8381 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:47.222593 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1d9aa3-1117-4993-bc20-c961539c8381 admin admin] PATH_INFO: `/v3/projects/478aae0d800b441f8f16231621a07ad1` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:47.223827 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1d9aa3-1117-4993-bc20-c961539c8381 admin admin] RBAC: Authorizing `identity:delete_project(project_id=478aae0d800b441f8f16231621a07ad1)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:47.225731 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1d9aa3-1117-4993-bc20-c961539c8381 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:47.273685 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-cd1d9aa3-1117-4993-bc20-c961539c8381 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '478aae0d800b441f8f16231621a07ad1', 'request_id': 'req-cd1d9aa3-1117-4993-bc20-c961539c8381'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:52:47.280040 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-cd1d9aa3-1117-4993-bc20-c961539c8381 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '478aae0d800b441f8f16231621a07ad1', 'request_id': 'req-cd1d9aa3-1117-4993-bc20-c961539c8381'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:52:47.285707 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-cd1d9aa3-1117-4993-bc20-c961539c8381 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '478aae0d800b441f8f16231621a07ad1', 'request_id': 'req-cd1d9aa3-1117-4993-bc20-c961539c8381'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:52:47.298377 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-cd1d9aa3-1117-4993-bc20-c961539c8381 admin admin] The token cache is being invalidate because project 478aae0d800b441f8f16231621a07ad1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80325) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 17:52:47.298670 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-cd1d9aa3-1117-4993-bc20-c961539c8381 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:52:47.299288 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-cd1d9aa3-1117-4993-bc20-c961539c8381 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:52:47.300780 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1250/2524] 10.222.0.89 () {58 vars in 1233 bytes} [Thu Jul 28 17:52:47 2022] DELETE /identity/v3/projects/478aae0d800b441f8f16231621a07ad1 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:52:47.305408 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b5af30-1147-4d1f-adde-c9ce20a7da99 service neutron] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:47.310273 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:47.316708 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b5af30-1147-4d1f-adde-c9ce20a7da99 service neutron] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:47.374212 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4d86de-766f-4ca3-9dcd-09b4a0a8172a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:47.374833 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4d86de-766f-4ca3-9dcd-09b4a0a8172a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:47.375109 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4d86de-766f-4ca3-9dcd-09b4a0a8172a admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:47.375380 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4d86de-766f-4ca3-9dcd-09b4a0a8172a admin admin] PATH_INFO: `/v3/users/e1c4c70a288447cbbd1bb9c7f1ca8a38` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:47.376536 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4d86de-766f-4ca3-9dcd-09b4a0a8172a admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1c4c70a288447cbbd1bb9c7f1ca8a38)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:47.378472 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4d86de-766f-4ca3-9dcd-09b4a0a8172a admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:47.424215 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-1f4d86de-766f-4ca3-9dcd-09b4a0a8172a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1c4c70a288447cbbd1bb9c7f1ca8a38'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:52:47.431621 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-1f4d86de-766f-4ca3-9dcd-09b4a0a8172a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1c4c70a288447cbbd1bb9c7f1ca8a38'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:52:47.432283 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-1f4d86de-766f-4ca3-9dcd-09b4a0a8172a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1c4c70a288447cbbd1bb9c7f1ca8a38'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:52:47.439969 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-1f4d86de-766f-4ca3-9dcd-09b4a0a8172a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1c4c70a288447cbbd1bb9c7f1ca8a38'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:52:47.453212 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-1f4d86de-766f-4ca3-9dcd-09b4a0a8172a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1c4c70a288447cbbd1bb9c7f1ca8a38'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:52:47.459194 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1275/2525] 10.222.0.89 () {58 vars in 1224 bytes} [Thu Jul 28 17:52:47 2022] DELETE /identity/v3/users/e1c4c70a288447cbbd1bb9c7f1ca8a38 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:52:47.727871 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1d9aa3-1117-4993-bc20-c961539c8381 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:47.729383 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-cd1d9aa3-1117-4993-bc20-c961539c8381 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:52:47.743952 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1d9aa3-1117-4993-bc20-c961539c8381 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:47.779163 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:47.805856 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cabb4f-39e6-4aff-9d0e-4d4eeb2cbe1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:47.806501 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cabb4f-39e6-4aff-9d0e-4d4eeb2cbe1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:47.806784 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cabb4f-39e6-4aff-9d0e-4d4eeb2cbe1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:47.807058 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cabb4f-39e6-4aff-9d0e-4d4eeb2cbe1b admin admin] PATH_INFO: `/v3/projects/4e1611fe4e2f42fa8eb3010b6bedce2d` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:47.808216 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cabb4f-39e6-4aff-9d0e-4d4eeb2cbe1b admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e1611fe4e2f42fa8eb3010b6bedce2d)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:47.810209 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cabb4f-39e6-4aff-9d0e-4d4eeb2cbe1b admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:47.857963 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-15cabb4f-39e6-4aff-9d0e-4d4eeb2cbe1b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e1611fe4e2f42fa8eb3010b6bedce2d', 'request_id': 'req-15cabb4f-39e6-4aff-9d0e-4d4eeb2cbe1b'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:52:47.864100 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-15cabb4f-39e6-4aff-9d0e-4d4eeb2cbe1b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e1611fe4e2f42fa8eb3010b6bedce2d', 'request_id': 'req-15cabb4f-39e6-4aff-9d0e-4d4eeb2cbe1b'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:52:47.870160 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-15cabb4f-39e6-4aff-9d0e-4d4eeb2cbe1b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e1611fe4e2f42fa8eb3010b6bedce2d', 'request_id': 'req-15cabb4f-39e6-4aff-9d0e-4d4eeb2cbe1b'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:52:47.883543 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-15cabb4f-39e6-4aff-9d0e-4d4eeb2cbe1b admin admin] The token cache is being invalidate because project 4e1611fe4e2f42fa8eb3010b6bedce2d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80325) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 17:52:47.883855 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-15cabb4f-39e6-4aff-9d0e-4d4eeb2cbe1b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:52:47.884479 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-15cabb4f-39e6-4aff-9d0e-4d4eeb2cbe1b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:52:47.886010 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1251/2526] 10.222.0.89 () {58 vars in 1233 bytes} [Thu Jul 28 17:52:47 2022] DELETE /identity/v3/projects/4e1611fe4e2f42fa8eb3010b6bedce2d => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:52:47.890801 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df72e7e-1c3e-4ed5-912f-89e25379da5d None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:47.891279 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df72e7e-1c3e-4ed5-912f-89e25379da5d None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:47.891379 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df72e7e-1c3e-4ed5-912f-89e25379da5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:47.912035 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-7df72e7e-1c3e-4ed5-912f-89e25379da5d None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:52:47.913295 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-7df72e7e-1c3e-4ed5-912f-89e25379da5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:52:47.971924 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1276/2527] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:52:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:52:47.981234 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec55d607-35d6-44f9-a900-958c29ed09b0 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:47.981522 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec55d607-35d6-44f9-a900-958c29ed09b0 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:47.981797 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec55d607-35d6-44f9-a900-958c29ed09b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:48.001083 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-ec55d607-35d6-44f9-a900-958c29ed09b0 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:52:48.002311 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-ec55d607-35d6-44f9-a900-958c29ed09b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:52:48.010745 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1252/2528] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:52:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:52:48.015049 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df72e7e-1c3e-4ed5-912f-89e25379da5d None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:48.026205 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df72e7e-1c3e-4ed5-912f-89e25379da5d None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:48.030725 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f981c0-b85f-4336-a4dd-bfd0c1bf3d49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:48.031326 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f981c0-b85f-4336-a4dd-bfd0c1bf3d49 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:48.031608 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f981c0-b85f-4336-a4dd-bfd0c1bf3d49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:48.031879 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f981c0-b85f-4336-a4dd-bfd0c1bf3d49 admin admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:48.032568 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f981c0-b85f-4336-a4dd-bfd0c1bf3d49 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:52:48.032869 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f981c0-b85f-4336-a4dd-bfd0c1bf3d49 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:48.034631 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f981c0-b85f-4336-a4dd-bfd0c1bf3d49 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:48.046521 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1277/2529] 10.222.0.89 () {58 vars in 1153 bytes} [Thu Jul 28 17:52:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:48.050802 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec55d607-35d6-44f9-a900-958c29ed09b0 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:48.060330 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec55d607-35d6-44f9-a900-958c29ed09b0 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:48.065040 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cf3539-ff70-4b12-bda0-4af42e62cc0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:48.065669 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cf3539-ff70-4b12-bda0-4af42e62cc0c admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:48.065944 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cf3539-ff70-4b12-bda0-4af42e62cc0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:48.066213 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cf3539-ff70-4b12-bda0-4af42e62cc0c admin admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:48.066864 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cf3539-ff70-4b12-bda0-4af42e62cc0c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:48.068724 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cf3539-ff70-4b12-bda0-4af42e62cc0c admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:48.097255 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1253/2530] 10.222.0.89 () {60 vars in 1153 bytes} [Thu Jul 28 17:52:48 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:52:48.101389 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f981c0-b85f-4336-a4dd-bfd0c1bf3d49 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:48.111680 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f981c0-b85f-4336-a4dd-bfd0c1bf3d49 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:48.116167 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:48.171433 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa16887e-54da-43ff-81a2-2afae28af65d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:48.172036 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa16887e-54da-43ff-81a2-2afae28af65d admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:48.172316 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa16887e-54da-43ff-81a2-2afae28af65d admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:48.172589 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa16887e-54da-43ff-81a2-2afae28af65d admin admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:48.173315 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa16887e-54da-43ff-81a2-2afae28af65d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:48.175229 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa16887e-54da-43ff-81a2-2afae28af65d admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:48.198093 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1278/2531] 10.222.0.89 () {60 vars in 1144 bytes} [Thu Jul 28 17:52:48 2022] POST /identity/v3/users => generated 293 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:52:48.202189 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cf3539-ff70-4b12-bda0-4af42e62cc0c admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:48.212944 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cf3539-ff70-4b12-bda0-4af42e62cc0c admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:48.217625 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8ef71f-f31b-4147-aa4a-4f0860480853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:48.218222 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8ef71f-f31b-4147-aa4a-4f0860480853 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:48.218502 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8ef71f-f31b-4147-aa4a-4f0860480853 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:48.218772 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8ef71f-f31b-4147-aa4a-4f0860480853 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:48.219456 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8ef71f-f31b-4147-aa4a-4f0860480853 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:48.221029 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8ef71f-f31b-4147-aa4a-4f0860480853 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:48.230969 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1254/2532] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:52:48 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 17:52:48.234980 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa16887e-54da-43ff-81a2-2afae28af65d admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:48.245656 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa16887e-54da-43ff-81a2-2afae28af65d admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:48.250606 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3e35b4-6074-4a22-8d6f-f0899f2e8016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:48.251400 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3e35b4-6074-4a22-8d6f-f0899f2e8016 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:48.251769 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3e35b4-6074-4a22-8d6f-f0899f2e8016 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:48.252074 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3e35b4-6074-4a22-8d6f-f0899f2e8016 admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:48.252791 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3e35b4-6074-4a22-8d6f-f0899f2e8016 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:48.254434 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3e35b4-6074-4a22-8d6f-f0899f2e8016 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:48.265292 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1279/2533] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:52:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:52:48.269297 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8ef71f-f31b-4147-aa4a-4f0860480853 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:48.279601 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8ef71f-f31b-4147-aa4a-4f0860480853 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:48.284276 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322de853-85bb-48b5-ac95-11aeed656539 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:48.284925 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322de853-85bb-48b5-ac95-11aeed656539 admin admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:48.285296 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322de853-85bb-48b5-ac95-11aeed656539 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:48.285571 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322de853-85bb-48b5-ac95-11aeed656539 admin admin] PATH_INFO: `/v3/projects/64e9086eb08b44559753091a4d981071/users/ff4db9a7c038412286a280da3bf5ee2d/roles/11f8731756fe4caca13efa61b71b11c5` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:48.300045 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322de853-85bb-48b5-ac95-11aeed656539 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64e9086eb08b44559753091a4d981071, user_id=ff4db9a7c038412286a280da3bf5ee2d, role_id=11f8731756fe4caca13efa61b71b11c5)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:48.302717 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322de853-85bb-48b5-ac95-11aeed656539 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:48.316089 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1255/2534] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:52:48 2022] PUT /identity/v3/projects/64e9086eb08b44559753091a4d981071/users/ff4db9a7c038412286a280da3bf5ee2d/roles/11f8731756fe4caca13efa61b71b11c5 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 17:52:48.327049 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55c6b47-41b5-472a-ae82-67164658de48 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:48.327329 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55c6b47-41b5-472a-ae82-67164658de48 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:48.327599 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55c6b47-41b5-472a-ae82-67164658de48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:48.355137 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-b55c6b47-41b5-472a-ae82-67164658de48 None None] MFA Rules not processed for user `ff4db9a7c038412286a280da3bf5ee2d`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:52:48.356385 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-b55c6b47-41b5-472a-ae82-67164658de48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:52:48.422035 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1280/2535] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:52:48 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:52:48.433885 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322de853-85bb-48b5-ac95-11aeed656539 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:48.435643 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-322de853-85bb-48b5-ac95-11aeed656539 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:52:48.503127 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:48.514443 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322de853-85bb-48b5-ac95-11aeed656539 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:48.518317 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5bb6e0-2f2c-470a-b3a0-56cd78bd5f40 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96d512d0c84240918dae6c4a3e8b7b79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:48.518919 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5bb6e0-2f2c-470a-b3a0-56cd78bd5f40 service cinder] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:48.519199 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5bb6e0-2f2c-470a-b3a0-56cd78bd5f40 service cinder] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:48.519469 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5bb6e0-2f2c-470a-b3a0-56cd78bd5f40 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:48.528255 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5bb6e0-2f2c-470a-b3a0-56cd78bd5f40 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:48.529708 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5bb6e0-2f2c-470a-b3a0-56cd78bd5f40 service cinder] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:48.539894 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1256/2536] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:52:48 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:52:48.560154 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55c6b47-41b5-472a-ae82-67164658de48 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:48.623919 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55c6b47-41b5-472a-ae82-67164658de48 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:48.628527 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebd660c-9178-4bb7-bf0e-ba6873bbba85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:48.629233 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebd660c-9178-4bb7-bf0e-ba6873bbba85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:48.629581 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebd660c-9178-4bb7-bf0e-ba6873bbba85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:48.629767 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebd660c-9178-4bb7-bf0e-ba6873bbba85 admin admin] PATH_INFO: `/v3/users/ff4db9a7c038412286a280da3bf5ee2d` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:48.630993 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebd660c-9178-4bb7-bf0e-ba6873bbba85 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff4db9a7c038412286a280da3bf5ee2d)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:48.633754 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebd660c-9178-4bb7-bf0e-ba6873bbba85 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:48.654966 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:48.682849 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-4ebd660c-9178-4bb7-bf0e-ba6873bbba85 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff4db9a7c038412286a280da3bf5ee2d'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:52:48.690096 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-4ebd660c-9178-4bb7-bf0e-ba6873bbba85 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff4db9a7c038412286a280da3bf5ee2d'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:52:48.690758 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-4ebd660c-9178-4bb7-bf0e-ba6873bbba85 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff4db9a7c038412286a280da3bf5ee2d'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:52:48.698106 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-4ebd660c-9178-4bb7-bf0e-ba6873bbba85 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff4db9a7c038412286a280da3bf5ee2d'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:52:48.709746 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-4ebd660c-9178-4bb7-bf0e-ba6873bbba85 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff4db9a7c038412286a280da3bf5ee2d'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:52:48.715927 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1281/2537] 10.222.0.89 () {58 vars in 1224 bytes} [Thu Jul 28 17:52:48 2022] DELETE /identity/v3/users/ff4db9a7c038412286a280da3bf5ee2d => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:52:48.720472 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66968235-0f3d-44be-a762-9ebd6ae177d1 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:48.720782 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66968235-0f3d-44be-a762-9ebd6ae177d1 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:48.721054 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66968235-0f3d-44be-a762-9ebd6ae177d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:48.740621 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-66968235-0f3d-44be-a762-9ebd6ae177d1 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:52:48.742906 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-66968235-0f3d-44be-a762-9ebd6ae177d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:52:48.802139 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1257/2538] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:52:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 17:52:48.812594 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebd660c-9178-4bb7-bf0e-ba6873bbba85 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:48.814133 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-4ebd660c-9178-4bb7-bf0e-ba6873bbba85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:52:48.871407 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebd660c-9178-4bb7-bf0e-ba6873bbba85 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:48.875330 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124f7db1-b228-49fd-80e6-971d607e8815 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76817cd541294331b0023069092e8e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:48.875990 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124f7db1-b228-49fd-80e6-971d607e8815 service neutron] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:48.876269 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124f7db1-b228-49fd-80e6-971d607e8815 service neutron] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:48.876540 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124f7db1-b228-49fd-80e6-971d607e8815 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:48.886851 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124f7db1-b228-49fd-80e6-971d607e8815 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:48.888276 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124f7db1-b228-49fd-80e6-971d607e8815 service neutron] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:48.897989 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1282/2539] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:52:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:52:49.171158 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66968235-0f3d-44be-a762-9ebd6ae177d1 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:49.172705 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-66968235-0f3d-44be-a762-9ebd6ae177d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:52:49.185815 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66968235-0f3d-44be-a762-9ebd6ae177d1 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:49.190618 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a0f5d2-fbb6-4cdd-94d4-ea3f7f4055e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:49.191261 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a0f5d2-fbb6-4cdd-94d4-ea3f7f4055e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:49.191544 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a0f5d2-fbb6-4cdd-94d4-ea3f7f4055e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:49.191819 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a0f5d2-fbb6-4cdd-94d4-ea3f7f4055e8 admin admin] PATH_INFO: `/v3/projects/64e9086eb08b44559753091a4d981071` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:49.192981 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a0f5d2-fbb6-4cdd-94d4-ea3f7f4055e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=64e9086eb08b44559753091a4d981071)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:49.195037 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a0f5d2-fbb6-4cdd-94d4-ea3f7f4055e8 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:49.242704 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-c2a0f5d2-fbb6-4cdd-94d4-ea3f7f4055e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64e9086eb08b44559753091a4d981071', 'request_id': 'req-c2a0f5d2-fbb6-4cdd-94d4-ea3f7f4055e8'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:52:49.250802 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-c2a0f5d2-fbb6-4cdd-94d4-ea3f7f4055e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64e9086eb08b44559753091a4d981071', 'request_id': 'req-c2a0f5d2-fbb6-4cdd-94d4-ea3f7f4055e8'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:52:49.256242 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-c2a0f5d2-fbb6-4cdd-94d4-ea3f7f4055e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64e9086eb08b44559753091a4d981071', 'request_id': 'req-c2a0f5d2-fbb6-4cdd-94d4-ea3f7f4055e8'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:52:49.268342 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-c2a0f5d2-fbb6-4cdd-94d4-ea3f7f4055e8 admin admin] The token cache is being invalidate because project 64e9086eb08b44559753091a4d981071 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80325) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 17:52:49.268628 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-c2a0f5d2-fbb6-4cdd-94d4-ea3f7f4055e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:52:49.269275 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-c2a0f5d2-fbb6-4cdd-94d4-ea3f7f4055e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:52:49.270766 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1258/2540] 10.222.0.89 () {58 vars in 1233 bytes} [Thu Jul 28 17:52:49 2022] DELETE /identity/v3/projects/64e9086eb08b44559753091a4d981071 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:52:49.276008 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a2fb71-74aa-4dfc-ad2c-fe9b501b48df None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:49.276326 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a2fb71-74aa-4dfc-ad2c-fe9b501b48df None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:49.276608 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a2fb71-74aa-4dfc-ad2c-fe9b501b48df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:49.296696 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-b9a2fb71-74aa-4dfc-ad2c-fe9b501b48df None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:52:49.297947 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-b9a2fb71-74aa-4dfc-ad2c-fe9b501b48df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:52:49.357471 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1283/2541] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:52:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:52:49.365330 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa649c5d-ef47-4301-81fc-361a50b89ca0 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:49.365615 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa649c5d-ef47-4301-81fc-361a50b89ca0 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:49.365886 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa649c5d-ef47-4301-81fc-361a50b89ca0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:49.386186 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-fa649c5d-ef47-4301-81fc-361a50b89ca0 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:52:49.387435 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-fa649c5d-ef47-4301-81fc-361a50b89ca0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:52:49.395533 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1259/2542] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:52:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:52:49.400244 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a2fb71-74aa-4dfc-ad2c-fe9b501b48df None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:49.410002 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a2fb71-74aa-4dfc-ad2c-fe9b501b48df None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:49.415354 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d5e195-b927-4a25-9335-d29ff382a446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:49.415947 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d5e195-b927-4a25-9335-d29ff382a446 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:49.416226 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d5e195-b927-4a25-9335-d29ff382a446 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:49.416497 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d5e195-b927-4a25-9335-d29ff382a446 admin admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:49.417215 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d5e195-b927-4a25-9335-d29ff382a446 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:52:49.417519 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d5e195-b927-4a25-9335-d29ff382a446 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:49.419101 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d5e195-b927-4a25-9335-d29ff382a446 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:49.431799 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1284/2543] 10.222.0.89 () {58 vars in 1153 bytes} [Thu Jul 28 17:52:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:52:49.436066 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa649c5d-ef47-4301-81fc-361a50b89ca0 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:49.445292 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa649c5d-ef47-4301-81fc-361a50b89ca0 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:49.450073 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8943340d-aecc-41a1-b0d5-37440894b5c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:49.450673 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8943340d-aecc-41a1-b0d5-37440894b5c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:49.450950 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8943340d-aecc-41a1-b0d5-37440894b5c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:49.451220 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8943340d-aecc-41a1-b0d5-37440894b5c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:49.451887 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8943340d-aecc-41a1-b0d5-37440894b5c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:49.453818 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8943340d-aecc-41a1-b0d5-37440894b5c5 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:49.481007 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1260/2544] 10.222.0.89 () {60 vars in 1153 bytes} [Thu Jul 28 17:52:49 2022] POST /identity/v3/projects => generated 371 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:52:49.485236 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d5e195-b927-4a25-9335-d29ff382a446 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:49.496498 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d5e195-b927-4a25-9335-d29ff382a446 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:49.520325 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:49.554915 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24376029-773c-4fa0-b9c6-c6d8cba40328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:49.555711 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24376029-773c-4fa0-b9c6-c6d8cba40328 admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:49.555989 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24376029-773c-4fa0-b9c6-c6d8cba40328 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:49.556258 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24376029-773c-4fa0-b9c6-c6d8cba40328 admin admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:49.556926 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24376029-773c-4fa0-b9c6-c6d8cba40328 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:49.558897 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24376029-773c-4fa0-b9c6-c6d8cba40328 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:49.578645 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1285/2545] 10.222.0.89 () {60 vars in 1144 bytes} [Thu Jul 28 17:52:49 2022] POST /identity/v3/users => generated 286 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:52:49.582788 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8943340d-aecc-41a1-b0d5-37440894b5c5 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:49.593330 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8943340d-aecc-41a1-b0d5-37440894b5c5 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:49.598015 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6530f6ab-63a5-42ae-b05b-7bdf2583a3fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:49.598616 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6530f6ab-63a5-42ae-b05b-7bdf2583a3fd admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:49.598892 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6530f6ab-63a5-42ae-b05b-7bdf2583a3fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:49.599155 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6530f6ab-63a5-42ae-b05b-7bdf2583a3fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:49.599846 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6530f6ab-63a5-42ae-b05b-7bdf2583a3fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:49.602467 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6530f6ab-63a5-42ae-b05b-7bdf2583a3fd admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:49.612678 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1261/2546] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:52:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:52:49.616573 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24376029-773c-4fa0-b9c6-c6d8cba40328 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:49.627410 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24376029-773c-4fa0-b9c6-c6d8cba40328 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:49.632362 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10051ed-f15c-4918-bae7-75e0f6758258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:49.633178 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10051ed-f15c-4918-bae7-75e0f6758258 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:49.633467 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10051ed-f15c-4918-bae7-75e0f6758258 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:49.633737 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10051ed-f15c-4918-bae7-75e0f6758258 admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:49.634423 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10051ed-f15c-4918-bae7-75e0f6758258 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:49.636036 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10051ed-f15c-4918-bae7-75e0f6758258 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:49.646618 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1286/2547] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:52:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:52:49.651502 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6530f6ab-63a5-42ae-b05b-7bdf2583a3fd admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:49.662403 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6530f6ab-63a5-42ae-b05b-7bdf2583a3fd admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:49.667580 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfab5f20-8627-4085-95e1-eea3bd0796d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:49.668241 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfab5f20-8627-4085-95e1-eea3bd0796d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:49.668518 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfab5f20-8627-4085-95e1-eea3bd0796d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:49.668794 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfab5f20-8627-4085-95e1-eea3bd0796d7 admin admin] PATH_INFO: `/v3/projects/51f7a6e9080d48418e4912be02e09922/users/aedeb9e3e967402aa1458ceedf023dac/roles/11f8731756fe4caca13efa61b71b11c5` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:49.685039 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfab5f20-8627-4085-95e1-eea3bd0796d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=51f7a6e9080d48418e4912be02e09922, user_id=aedeb9e3e967402aa1458ceedf023dac, role_id=11f8731756fe4caca13efa61b71b11c5)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:49.687791 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfab5f20-8627-4085-95e1-eea3bd0796d7 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:49.701173 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1262/2548] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:52:49 2022] PUT /identity/v3/projects/51f7a6e9080d48418e4912be02e09922/users/aedeb9e3e967402aa1458ceedf023dac/roles/11f8731756fe4caca13efa61b71b11c5 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:52:49.706263 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaec9e9-1175-47f5-9f15-6b42d792cee9 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:49.706644 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaec9e9-1175-47f5-9f15-6b42d792cee9 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:49.706948 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaec9e9-1175-47f5-9f15-6b42d792cee9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:49.726048 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-dfaec9e9-1175-47f5-9f15-6b42d792cee9 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:52:49.727942 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-dfaec9e9-1175-47f5-9f15-6b42d792cee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:52:49.785969 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1287/2549] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:52:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:52:49.796221 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfab5f20-8627-4085-95e1-eea3bd0796d7 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:49.797903 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-cfab5f20-8627-4085-95e1-eea3bd0796d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:52:49.804439 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:52:49.857082 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfab5f20-8627-4085-95e1-eea3bd0796d7 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:49.860679 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02f38ba-b2a4-4b62-aa79-8aea8cc19a39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76817cd541294331b0023069092e8e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:49.861330 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02f38ba-b2a4-4b62-aa79-8aea8cc19a39 service neutron] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:49.861607 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02f38ba-b2a4-4b62-aa79-8aea8cc19a39 service neutron] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:49.861873 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02f38ba-b2a4-4b62-aa79-8aea8cc19a39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:49.870982 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02f38ba-b2a4-4b62-aa79-8aea8cc19a39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:49.872407 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02f38ba-b2a4-4b62-aa79-8aea8cc19a39 service neutron] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:49.881938 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1263/2550] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:52:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:52:50.529429 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b4e915-dd6b-41b2-88a1-0aaa39e181e2 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:50.529982 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b4e915-dd6b-41b2-88a1-0aaa39e181e2 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:50.529982 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b4e915-dd6b-41b2-88a1-0aaa39e181e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:50.560579 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-76b4e915-dd6b-41b2-88a1-0aaa39e181e2 None None] MFA Rules not processed for user `aedeb9e3e967402aa1458ceedf023dac`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:52:50.561841 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-76b4e915-dd6b-41b2-88a1-0aaa39e181e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:52:50.627946 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1288/2551] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:52:50 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:52:50.639808 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02f38ba-b2a4-4b62-aa79-8aea8cc19a39 service neutron] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:52:50.641399 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-d02f38ba-b2a4-4b62-aa79-8aea8cc19a39 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:52:50.700920 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02f38ba-b2a4-4b62-aa79-8aea8cc19a39 service neutron] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:52:50.704578 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8866b4aa-fb18-4753-a21d-8824391cc989 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96d512d0c84240918dae6c4a3e8b7b79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:52:50.705223 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8866b4aa-fb18-4753-a21d-8824391cc989 service cinder] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:50.705506 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8866b4aa-fb18-4753-a21d-8824391cc989 service cinder] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:50.705781 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8866b4aa-fb18-4753-a21d-8824391cc989 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:50.714685 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8866b4aa-fb18-4753-a21d-8824391cc989 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:52:50.716108 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8866b4aa-fb18-4753-a21d-8824391cc989 service cinder] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:52:50.725036 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1264/2552] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:52:50 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:52:52.085425 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad41b014-9bd2-413b-9534-764a34f2917c None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:52:52.085921 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad41b014-9bd2-413b-9534-764a34f2917c None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:52:52.085921 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad41b014-9bd2-413b-9534-764a34f2917c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:52:52.105458 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-ad41b014-9bd2-413b-9534-764a34f2917c None None] MFA Rules not processed for user `aedeb9e3e967402aa1458ceedf023dac`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:52:52.106685 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-ad41b014-9bd2-413b-9534-764a34f2917c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:52:52.115616 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1289/2553] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:52:52 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:53:00.721103 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=50 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:02.099749 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=84,keystone:INSERT=3 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:08.702873 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8866b4aa-fb18-4753-a21d-8824391cc989 service cinder] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:08.704465 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-8866b4aa-fb18-4753-a21d-8824391cc989 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:08.717873 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8866b4aa-fb18-4753-a21d-8824391cc989 service cinder] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:08.721643 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957f1a94-15a7-4a97-a17e-44d4eaf22d35 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76817cd541294331b0023069092e8e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:08.722250 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957f1a94-15a7-4a97-a17e-44d4eaf22d35 service neutron] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:08.722538 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957f1a94-15a7-4a97-a17e-44d4eaf22d35 service neutron] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:08.722814 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957f1a94-15a7-4a97-a17e-44d4eaf22d35 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:08.724549 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-957f1a94-15a7-4a97-a17e-44d4eaf22d35 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:08.737762 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957f1a94-15a7-4a97-a17e-44d4eaf22d35 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:08.739167 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957f1a94-15a7-4a97-a17e-44d4eaf22d35 service neutron] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:08.749449 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1265/2554] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:53:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:53:08.861540 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b11d54c-d1c4-45ac-b1d1-1a653c80d6c7 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:08.862092 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b11d54c-d1c4-45ac-b1d1-1a653c80d6c7 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:08.862092 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b11d54c-d1c4-45ac-b1d1-1a653c80d6c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:08.882446 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-0b11d54c-d1c4-45ac-b1d1-1a653c80d6c7 None None] MFA Rules not processed for user `aedeb9e3e967402aa1458ceedf023dac`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:53:08.883754 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-0b11d54c-d1c4-45ac-b1d1-1a653c80d6c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:08.892692 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1290/2555] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:53:08 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:53:12.471691 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957f1a94-15a7-4a97-a17e-44d4eaf22d35 service neutron] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:12.473797 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-957f1a94-15a7-4a97-a17e-44d4eaf22d35 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:12.490323 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957f1a94-15a7-4a97-a17e-44d4eaf22d35 service neutron] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:12.496317 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc83f6e-9529-4c32-a700-0b0b8f5b1b37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:12.496997 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc83f6e-9529-4c32-a700-0b0b8f5b1b37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:12.497309 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc83f6e-9529-4c32-a700-0b0b8f5b1b37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:12.497583 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc83f6e-9529-4c32-a700-0b0b8f5b1b37 admin admin] PATH_INFO: `/v3/users/0b97c60c0360466fb878e1e9141e1f3e` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:12.498790 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc83f6e-9529-4c32-a700-0b0b8f5b1b37 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b97c60c0360466fb878e1e9141e1f3e)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:12.500711 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc83f6e-9529-4c32-a700-0b0b8f5b1b37 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:12.554711 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-6bc83f6e-9529-4c32-a700-0b0b8f5b1b37 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b97c60c0360466fb878e1e9141e1f3e'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:53:12.562848 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-6bc83f6e-9529-4c32-a700-0b0b8f5b1b37 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b97c60c0360466fb878e1e9141e1f3e'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:53:12.570361 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-6bc83f6e-9529-4c32-a700-0b0b8f5b1b37 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b97c60c0360466fb878e1e9141e1f3e'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:53:12.570981 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-6bc83f6e-9529-4c32-a700-0b0b8f5b1b37 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b97c60c0360466fb878e1e9141e1f3e'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:53:12.571591 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-6bc83f6e-9529-4c32-a700-0b0b8f5b1b37 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b97c60c0360466fb878e1e9141e1f3e'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:53:12.595891 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1266/2556] 10.222.0.89 () {58 vars in 1224 bytes} [Thu Jul 28 17:53:12 2022] DELETE /identity/v3/users/0b97c60c0360466fb878e1e9141e1f3e => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:53:12.814107 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b11d54c-d1c4-45ac-b1d1-1a653c80d6c7 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:12.834912 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b11d54c-d1c4-45ac-b1d1-1a653c80d6c7 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:12.916201 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99df4ecf-4865-4341-9155-4b9241d48058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:12.917250 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99df4ecf-4865-4341-9155-4b9241d48058 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:12.917569 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99df4ecf-4865-4341-9155-4b9241d48058 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:12.917842 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99df4ecf-4865-4341-9155-4b9241d48058 admin admin] PATH_INFO: `/v3/projects/65535f9817c448dfac997e93efa1a909` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:12.919056 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99df4ecf-4865-4341-9155-4b9241d48058 admin admin] RBAC: Authorizing `identity:delete_project(project_id=65535f9817c448dfac997e93efa1a909)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:12.920969 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99df4ecf-4865-4341-9155-4b9241d48058 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:12.986927 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-99df4ecf-4865-4341-9155-4b9241d48058 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65535f9817c448dfac997e93efa1a909', 'request_id': 'req-99df4ecf-4865-4341-9155-4b9241d48058'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:53:12.994164 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-99df4ecf-4865-4341-9155-4b9241d48058 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65535f9817c448dfac997e93efa1a909', 'request_id': 'req-99df4ecf-4865-4341-9155-4b9241d48058'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:53:13.001505 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-99df4ecf-4865-4341-9155-4b9241d48058 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65535f9817c448dfac997e93efa1a909', 'request_id': 'req-99df4ecf-4865-4341-9155-4b9241d48058'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:53:13.016658 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-99df4ecf-4865-4341-9155-4b9241d48058 admin admin] The token cache is being invalidate because project 65535f9817c448dfac997e93efa1a909 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80324) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 17:53:13.017059 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-99df4ecf-4865-4341-9155-4b9241d48058 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:53:13.017789 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-99df4ecf-4865-4341-9155-4b9241d48058 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:53:13.019385 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1291/2557] 10.222.0.89 () {58 vars in 1233 bytes} [Thu Jul 28 17:53:12 2022] DELETE /identity/v3/projects/65535f9817c448dfac997e93efa1a909 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:53:13.024658 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c72b665-9a6f-4f38-a609-e0b4d66e037d None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:13.025816 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c72b665-9a6f-4f38-a609-e0b4d66e037d None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:13.026113 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c72b665-9a6f-4f38-a609-e0b4d66e037d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:13.049092 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-3c72b665-9a6f-4f38-a609-e0b4d66e037d None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:53:13.050472 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-3c72b665-9a6f-4f38-a609-e0b4d66e037d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:13.120393 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1267/2558] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:53:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:53:13.129204 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11ae56d-a61f-4da9-a8fb-3cae813e48a3 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:13.130107 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11ae56d-a61f-4da9-a8fb-3cae813e48a3 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:13.130561 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11ae56d-a61f-4da9-a8fb-3cae813e48a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:13.155254 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-c11ae56d-a61f-4da9-a8fb-3cae813e48a3 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:53:13.156541 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-c11ae56d-a61f-4da9-a8fb-3cae813e48a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:13.164791 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1292/2559] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:53:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:53:13.170126 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c72b665-9a6f-4f38-a609-e0b4d66e037d None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:13.180002 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c72b665-9a6f-4f38-a609-e0b4d66e037d None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:13.185388 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f33309-b012-44b4-b82c-9188dd17f7ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:13.186028 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f33309-b012-44b4-b82c-9188dd17f7ba admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:13.186401 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f33309-b012-44b4-b82c-9188dd17f7ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:13.186676 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f33309-b012-44b4-b82c-9188dd17f7ba admin admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:13.187389 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f33309-b012-44b4-b82c-9188dd17f7ba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:13.187693 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f33309-b012-44b4-b82c-9188dd17f7ba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:13.189329 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f33309-b012-44b4-b82c-9188dd17f7ba admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:13.202027 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1268/2560] 10.222.0.89 () {58 vars in 1153 bytes} [Thu Jul 28 17:53:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:53:13.206522 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11ae56d-a61f-4da9-a8fb-3cae813e48a3 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:13.216821 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11ae56d-a61f-4da9-a8fb-3cae813e48a3 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:13.221782 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ca1831-8765-4df0-ab64-88731007b5af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:13.222415 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ca1831-8765-4df0-ab64-88731007b5af admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:13.222699 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ca1831-8765-4df0-ab64-88731007b5af admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:13.222975 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ca1831-8765-4df0-ab64-88731007b5af admin admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:13.223848 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ca1831-8765-4df0-ab64-88731007b5af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:13.225807 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ca1831-8765-4df0-ab64-88731007b5af admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:13.256731 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1293/2561] 10.222.0.89 () {60 vars in 1153 bytes} [Thu Jul 28 17:53:13 2022] POST /identity/v3/projects => generated 367 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:53:13.261344 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f33309-b012-44b4-b82c-9188dd17f7ba admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:13.271981 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f33309-b012-44b4-b82c-9188dd17f7ba admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:13.323043 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:13.342732 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1097920-9fcb-42a2-bf0a-fb8c40a0c0a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:13.343353 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1097920-9fcb-42a2-bf0a-fb8c40a0c0a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:13.343576 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1097920-9fcb-42a2-bf0a-fb8c40a0c0a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:13.344134 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1097920-9fcb-42a2-bf0a-fb8c40a0c0a3 admin admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:13.344527 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1097920-9fcb-42a2-bf0a-fb8c40a0c0a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:13.346506 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1097920-9fcb-42a2-bf0a-fb8c40a0c0a3 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:13.371968 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1269/2562] 10.222.0.89 () {60 vars in 1144 bytes} [Thu Jul 28 17:53:13 2022] POST /identity/v3/users => generated 284 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:53:13.376303 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ca1831-8765-4df0-ab64-88731007b5af admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:13.388090 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ca1831-8765-4df0-ab64-88731007b5af admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:13.393384 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc9cc9e-176d-484f-8890-a08a67e3649a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:13.394002 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc9cc9e-176d-484f-8890-a08a67e3649a admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:13.394288 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc9cc9e-176d-484f-8890-a08a67e3649a admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:13.394571 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc9cc9e-176d-484f-8890-a08a67e3649a admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:13.395264 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc9cc9e-176d-484f-8890-a08a67e3649a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:13.396855 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc9cc9e-176d-484f-8890-a08a67e3649a admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:13.415324 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1294/2563] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:53:13 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:53:13.416642 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1097920-9fcb-42a2-bf0a-fb8c40a0c0a3 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:13.430226 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1097920-9fcb-42a2-bf0a-fb8c40a0c0a3 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:13.436012 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72427bed-80eb-4b12-9d2f-5e9ce29ab43d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:13.436657 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72427bed-80eb-4b12-9d2f-5e9ce29ab43d admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:13.436918 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72427bed-80eb-4b12-9d2f-5e9ce29ab43d admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:13.437246 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72427bed-80eb-4b12-9d2f-5e9ce29ab43d admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:13.437969 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72427bed-80eb-4b12-9d2f-5e9ce29ab43d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:13.439598 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72427bed-80eb-4b12-9d2f-5e9ce29ab43d admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:13.452162 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1270/2564] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:53:13 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 17:53:13.456841 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc9cc9e-176d-484f-8890-a08a67e3649a admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:13.477575 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc9cc9e-176d-484f-8890-a08a67e3649a admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:13.483477 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161dcfd4-d20c-4987-bced-7796c0dc0f07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:13.484273 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161dcfd4-d20c-4987-bced-7796c0dc0f07 admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:13.484656 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161dcfd4-d20c-4987-bced-7796c0dc0f07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:13.485169 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161dcfd4-d20c-4987-bced-7796c0dc0f07 admin admin] PATH_INFO: `/v3/projects/bc5740fc21e04cf79756c0e0e8a77e6a/users/9827d95cc53a46d39b2ca83075bb9154/roles/11f8731756fe4caca13efa61b71b11c5` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:13.501515 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161dcfd4-d20c-4987-bced-7796c0dc0f07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc5740fc21e04cf79756c0e0e8a77e6a, user_id=9827d95cc53a46d39b2ca83075bb9154, role_id=11f8731756fe4caca13efa61b71b11c5)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:13.504280 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161dcfd4-d20c-4987-bced-7796c0dc0f07 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:13.539121 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b92bf4-8103-4c15-a0af-dc4a6483f7c9 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:13.539678 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b92bf4-8103-4c15-a0af-dc4a6483f7c9 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:13.539678 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b92bf4-8103-4c15-a0af-dc4a6483f7c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:13.543035 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1295/2565] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:53:13 2022] PUT /identity/v3/projects/bc5740fc21e04cf79756c0e0e8a77e6a/users/9827d95cc53a46d39b2ca83075bb9154/roles/11f8731756fe4caca13efa61b71b11c5 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:53:13.572872 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-31b92bf4-8103-4c15-a0af-dc4a6483f7c9 None None] MFA Rules not processed for user `9827d95cc53a46d39b2ca83075bb9154`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:53:13.574149 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-31b92bf4-8103-4c15-a0af-dc4a6483f7c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:13.641047 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1271/2566] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:53:13 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:53:13.652597 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161dcfd4-d20c-4987-bced-7796c0dc0f07 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:13.654364 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-161dcfd4-d20c-4987-bced-7796c0dc0f07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:13.676382 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:13.717531 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161dcfd4-d20c-4987-bced-7796c0dc0f07 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:13.721257 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3342688-b35a-42d2-bbfb-0c3518956a83 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96d512d0c84240918dae6c4a3e8b7b79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:13.722129 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3342688-b35a-42d2-bbfb-0c3518956a83 service cinder] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:13.722402 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3342688-b35a-42d2-bbfb-0c3518956a83 service cinder] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:13.722673 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3342688-b35a-42d2-bbfb-0c3518956a83 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:13.731085 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3342688-b35a-42d2-bbfb-0c3518956a83 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:13.732502 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3342688-b35a-42d2-bbfb-0c3518956a83 service cinder] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:13.741575 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1296/2567] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:53:13 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:53:22.492245 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b92bf4-8103-4c15-a0af-dc4a6483f7c9 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:22.557640 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b92bf4-8103-4c15-a0af-dc4a6483f7c9 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:22.562261 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ce1a34-9c52-4815-aed1-84434e2d1f48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:22.562902 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ce1a34-9c52-4815-aed1-84434e2d1f48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:22.563181 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ce1a34-9c52-4815-aed1-84434e2d1f48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:22.563453 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ce1a34-9c52-4815-aed1-84434e2d1f48 admin admin] PATH_INFO: `/v3/users/9827d95cc53a46d39b2ca83075bb9154` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:22.564621 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ce1a34-9c52-4815-aed1-84434e2d1f48 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9827d95cc53a46d39b2ca83075bb9154)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:22.566579 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ce1a34-9c52-4815-aed1-84434e2d1f48 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:22.612058 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-d6ce1a34-9c52-4815-aed1-84434e2d1f48 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9827d95cc53a46d39b2ca83075bb9154'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:53:22.618858 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-d6ce1a34-9c52-4815-aed1-84434e2d1f48 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9827d95cc53a46d39b2ca83075bb9154'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:53:22.626889 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-d6ce1a34-9c52-4815-aed1-84434e2d1f48 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9827d95cc53a46d39b2ca83075bb9154'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:53:22.627576 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-d6ce1a34-9c52-4815-aed1-84434e2d1f48 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9827d95cc53a46d39b2ca83075bb9154'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:53:22.628134 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:22.628965 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-d6ce1a34-9c52-4815-aed1-84434e2d1f48 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9827d95cc53a46d39b2ca83075bb9154'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:53:22.653027 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1272/2568] 10.222.0.89 () {58 vars in 1224 bytes} [Thu Jul 28 17:53:22 2022] DELETE /identity/v3/users/9827d95cc53a46d39b2ca83075bb9154 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:53:22.657868 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f087811-0cad-4b69-8b22-9df688fb18ad None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:22.658504 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f087811-0cad-4b69-8b22-9df688fb18ad None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:22.658725 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f087811-0cad-4b69-8b22-9df688fb18ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:22.679695 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-5f087811-0cad-4b69-8b22-9df688fb18ad None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:53:22.680899 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-5f087811-0cad-4b69-8b22-9df688fb18ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:22.754154 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1297/2569] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:53:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:53:22.763546 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ce1a34-9c52-4815-aed1-84434e2d1f48 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:22.764964 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-d6ce1a34-9c52-4815-aed1-84434e2d1f48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:22.836626 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ce1a34-9c52-4815-aed1-84434e2d1f48 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:22.841085 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9151da09-ff0f-46d5-9eee-0ce93d8a04d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76817cd541294331b0023069092e8e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:22.842520 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9151da09-ff0f-46d5-9eee-0ce93d8a04d9 service neutron] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:22.842520 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9151da09-ff0f-46d5-9eee-0ce93d8a04d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:22.842520 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9151da09-ff0f-46d5-9eee-0ce93d8a04d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:22.852330 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9151da09-ff0f-46d5-9eee-0ce93d8a04d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:22.854366 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9151da09-ff0f-46d5-9eee-0ce93d8a04d9 service neutron] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:22.864501 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1273/2570] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:53:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:53:23.184023 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f087811-0cad-4b69-8b22-9df688fb18ad None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:23.185577 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-5f087811-0cad-4b69-8b22-9df688fb18ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:23.199503 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f087811-0cad-4b69-8b22-9df688fb18ad None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:23.204208 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732a221b-8033-4c08-8af0-9414d2073d4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:23.204868 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732a221b-8033-4c08-8af0-9414d2073d4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:23.205185 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732a221b-8033-4c08-8af0-9414d2073d4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:23.205454 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732a221b-8033-4c08-8af0-9414d2073d4e admin admin] PATH_INFO: `/v3/projects/bc5740fc21e04cf79756c0e0e8a77e6a` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:23.206668 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732a221b-8033-4c08-8af0-9414d2073d4e admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc5740fc21e04cf79756c0e0e8a77e6a)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:23.208587 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732a221b-8033-4c08-8af0-9414d2073d4e admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:23.256901 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-732a221b-8033-4c08-8af0-9414d2073d4e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc5740fc21e04cf79756c0e0e8a77e6a', 'request_id': 'req-732a221b-8033-4c08-8af0-9414d2073d4e'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:53:23.263878 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-732a221b-8033-4c08-8af0-9414d2073d4e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc5740fc21e04cf79756c0e0e8a77e6a', 'request_id': 'req-732a221b-8033-4c08-8af0-9414d2073d4e'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:53:23.270009 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-732a221b-8033-4c08-8af0-9414d2073d4e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc5740fc21e04cf79756c0e0e8a77e6a', 'request_id': 'req-732a221b-8033-4c08-8af0-9414d2073d4e'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:53:23.283357 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-732a221b-8033-4c08-8af0-9414d2073d4e admin admin] The token cache is being invalidate because project bc5740fc21e04cf79756c0e0e8a77e6a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80324) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 17:53:23.283646 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-732a221b-8033-4c08-8af0-9414d2073d4e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:53:23.284261 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-732a221b-8033-4c08-8af0-9414d2073d4e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:53:23.285793 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1298/2571] 10.222.0.89 () {58 vars in 1233 bytes} [Thu Jul 28 17:53:23 2022] DELETE /identity/v3/projects/bc5740fc21e04cf79756c0e0e8a77e6a => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:53:23.291673 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3c3934-0c32-440c-8f62-6eb3af91ac0c None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:23.291963 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3c3934-0c32-440c-8f62-6eb3af91ac0c None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:23.292245 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3c3934-0c32-440c-8f62-6eb3af91ac0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:23.311274 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-ba3c3934-0c32-440c-8f62-6eb3af91ac0c None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:53:23.312488 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-ba3c3934-0c32-440c-8f62-6eb3af91ac0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:23.371254 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1274/2572] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:53:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 17:53:23.379158 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e149cf-5ae7-4a5f-8454-999466fbe2f6 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:23.379379 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e149cf-5ae7-4a5f-8454-999466fbe2f6 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:23.379650 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e149cf-5ae7-4a5f-8454-999466fbe2f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:23.402239 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-e9e149cf-5ae7-4a5f-8454-999466fbe2f6 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:53:23.403470 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-e9e149cf-5ae7-4a5f-8454-999466fbe2f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:23.411496 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1299/2573] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:53:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:53:23.415939 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3c3934-0c32-440c-8f62-6eb3af91ac0c None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:23.426103 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3c3934-0c32-440c-8f62-6eb3af91ac0c None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:23.431387 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cedcf42-a348-45c6-b995-a333d594d6a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:23.432585 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cedcf42-a348-45c6-b995-a333d594d6a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:23.433114 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cedcf42-a348-45c6-b995-a333d594d6a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:23.433114 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cedcf42-a348-45c6-b995-a333d594d6a2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:23.433798 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cedcf42-a348-45c6-b995-a333d594d6a2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:23.434103 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cedcf42-a348-45c6-b995-a333d594d6a2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:23.435698 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cedcf42-a348-45c6-b995-a333d594d6a2 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:23.449092 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1275/2574] 10.222.0.89 () {58 vars in 1153 bytes} [Thu Jul 28 17:53:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:23.453635 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e149cf-5ae7-4a5f-8454-999466fbe2f6 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:23.463458 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e149cf-5ae7-4a5f-8454-999466fbe2f6 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:23.468668 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0749cfd-cb64-40fb-ac6d-fe31d1f23d0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:23.469381 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0749cfd-cb64-40fb-ac6d-fe31d1f23d0e admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:23.469661 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0749cfd-cb64-40fb-ac6d-fe31d1f23d0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:23.469933 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0749cfd-cb64-40fb-ac6d-fe31d1f23d0e admin admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:23.470674 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0749cfd-cb64-40fb-ac6d-fe31d1f23d0e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:23.472619 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0749cfd-cb64-40fb-ac6d-fe31d1f23d0e admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:23.501105 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1300/2575] 10.222.0.89 () {60 vars in 1153 bytes} [Thu Jul 28 17:53:23 2022] POST /identity/v3/projects => generated 387 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 17:53:23.505275 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cedcf42-a348-45c6-b995-a333d594d6a2 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:23.516414 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cedcf42-a348-45c6-b995-a333d594d6a2 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:23.573279 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10519291-e4ab-49c6-8faa-5c494e64e63c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:23.573899 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10519291-e4ab-49c6-8faa-5c494e64e63c admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:23.574314 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10519291-e4ab-49c6-8faa-5c494e64e63c admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:23.574590 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10519291-e4ab-49c6-8faa-5c494e64e63c admin admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:23.575263 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10519291-e4ab-49c6-8faa-5c494e64e63c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:23.577206 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10519291-e4ab-49c6-8faa-5c494e64e63c admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:23.586956 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:INSERT=3,keystone:DELETE=3,keystone:SELECT=94 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:23.600341 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1276/2576] 10.222.0.89 () {60 vars in 1144 bytes} [Thu Jul 28 17:53:23 2022] POST /identity/v3/users => generated 294 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 17:53:23.604717 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0749cfd-cb64-40fb-ac6d-fe31d1f23d0e admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:23.615101 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0749cfd-cb64-40fb-ac6d-fe31d1f23d0e admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:23.620810 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb55a8cb-ffbe-4a2b-8497-c592aff5131b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:23.621470 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb55a8cb-ffbe-4a2b-8497-c592aff5131b admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:23.621756 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb55a8cb-ffbe-4a2b-8497-c592aff5131b admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:23.622037 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb55a8cb-ffbe-4a2b-8497-c592aff5131b admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:23.622732 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb55a8cb-ffbe-4a2b-8497-c592aff5131b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:23.624326 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb55a8cb-ffbe-4a2b-8497-c592aff5131b admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:23.634864 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1301/2577] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:53:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:53:23.640026 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10519291-e4ab-49c6-8faa-5c494e64e63c admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:23.650114 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10519291-e4ab-49c6-8faa-5c494e64e63c admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:23.655175 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedabed6-99ef-4608-b67d-1cfd3dcf9269 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:23.655778 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedabed6-99ef-4608-b67d-1cfd3dcf9269 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:23.656060 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedabed6-99ef-4608-b67d-1cfd3dcf9269 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:23.656334 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedabed6-99ef-4608-b67d-1cfd3dcf9269 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:23.657028 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedabed6-99ef-4608-b67d-1cfd3dcf9269 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:23.658665 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedabed6-99ef-4608-b67d-1cfd3dcf9269 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:23.670015 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1277/2578] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:53:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:53:23.674107 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb55a8cb-ffbe-4a2b-8497-c592aff5131b admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:23.684960 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb55a8cb-ffbe-4a2b-8497-c592aff5131b admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:23.689860 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc83c2ae-03bc-40b2-8ab8-d6aaf0993564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:23.690506 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc83c2ae-03bc-40b2-8ab8-d6aaf0993564 admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:23.690784 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc83c2ae-03bc-40b2-8ab8-d6aaf0993564 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:23.691056 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc83c2ae-03bc-40b2-8ab8-d6aaf0993564 admin admin] PATH_INFO: `/v3/projects/814856f317024d35ae0f46fce85a7ec3/users/d77724fb74a74918a275f81c75ad0ea1/roles/11f8731756fe4caca13efa61b71b11c5` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:23.693269 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:23.709851 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc83c2ae-03bc-40b2-8ab8-d6aaf0993564 admin admin] RBAC: Authorizing `identity:create_grant(project_id=814856f317024d35ae0f46fce85a7ec3, user_id=d77724fb74a74918a275f81c75ad0ea1, role_id=11f8731756fe4caca13efa61b71b11c5)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:23.712644 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc83c2ae-03bc-40b2-8ab8-d6aaf0993564 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:23.726989 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1302/2579] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:53:23 2022] PUT /identity/v3/projects/814856f317024d35ae0f46fce85a7ec3/users/d77724fb74a74918a275f81c75ad0ea1/roles/11f8731756fe4caca13efa61b71b11c5 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:53:23.739678 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e723ed61-a273-4939-a61f-0220754c8121 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:23.739960 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e723ed61-a273-4939-a61f-0220754c8121 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:23.740235 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e723ed61-a273-4939-a61f-0220754c8121 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:23.768344 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-e723ed61-a273-4939-a61f-0220754c8121 None None] MFA Rules not processed for user `d77724fb74a74918a275f81c75ad0ea1`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:53:23.769578 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-e723ed61-a273-4939-a61f-0220754c8121 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:23.837233 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1278/2580] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:53:23 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:53:23.841927 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc83c2ae-03bc-40b2-8ab8-d6aaf0993564 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:23.910541 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc83c2ae-03bc-40b2-8ab8-d6aaf0993564 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:23.915654 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423a14a9-1c91-4ec1-a18a-e1193552b9fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:23.917619 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423a14a9-1c91-4ec1-a18a-e1193552b9fd admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:23.917812 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423a14a9-1c91-4ec1-a18a-e1193552b9fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:23.918059 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423a14a9-1c91-4ec1-a18a-e1193552b9fd admin admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:23.918862 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423a14a9-1c91-4ec1-a18a-e1193552b9fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:23.920651 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423a14a9-1c91-4ec1-a18a-e1193552b9fd admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:23.953203 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1303/2581] 10.222.0.89 () {60 vars in 1153 bytes} [Thu Jul 28 17:53:23 2022] POST /identity/v3/projects => generated 387 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:53:23.957427 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e723ed61-a273-4939-a61f-0220754c8121 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:23.969454 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e723ed61-a273-4939-a61f-0220754c8121 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:24.038759 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c732a80-8fdf-43ae-95ee-d3e9cccd1290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:24.039386 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c732a80-8fdf-43ae-95ee-d3e9cccd1290 admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:24.039661 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c732a80-8fdf-43ae-95ee-d3e9cccd1290 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:24.039956 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c732a80-8fdf-43ae-95ee-d3e9cccd1290 admin admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:24.041614 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c732a80-8fdf-43ae-95ee-d3e9cccd1290 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:24.043508 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c732a80-8fdf-43ae-95ee-d3e9cccd1290 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:24.065301 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1279/2582] 10.222.0.89 () {60 vars in 1144 bytes} [Thu Jul 28 17:53:23 2022] POST /identity/v3/users => generated 300 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:53:24.069285 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423a14a9-1c91-4ec1-a18a-e1193552b9fd admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:24.080204 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423a14a9-1c91-4ec1-a18a-e1193552b9fd admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:24.086845 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd0e0a5-fc10-43f0-81ba-d1fceed5c692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:24.087462 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd0e0a5-fc10-43f0-81ba-d1fceed5c692 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:24.087744 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd0e0a5-fc10-43f0-81ba-d1fceed5c692 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:24.088142 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd0e0a5-fc10-43f0-81ba-d1fceed5c692 admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:24.088852 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd0e0a5-fc10-43f0-81ba-d1fceed5c692 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:24.090511 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd0e0a5-fc10-43f0-81ba-d1fceed5c692 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:24.101808 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1304/2583] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:53:24 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:53:24.106669 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c732a80-8fdf-43ae-95ee-d3e9cccd1290 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:24.116627 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c732a80-8fdf-43ae-95ee-d3e9cccd1290 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:24.121917 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aca046a-d7a7-4c31-96ae-fda5152e77e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:24.122614 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aca046a-d7a7-4c31-96ae-fda5152e77e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:24.122897 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aca046a-d7a7-4c31-96ae-fda5152e77e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:24.123169 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aca046a-d7a7-4c31-96ae-fda5152e77e9 admin admin] PATH_INFO: `/v3/projects/a24b249eaef84f608e2f2bffe2452728/users/e960e6b09a7947e682ee7c7aab06d669/roles/73e457231a824ea98c28d22610be640c` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:24.140030 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aca046a-d7a7-4c31-96ae-fda5152e77e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a24b249eaef84f608e2f2bffe2452728, user_id=e960e6b09a7947e682ee7c7aab06d669, role_id=73e457231a824ea98c28d22610be640c)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:24.142808 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aca046a-d7a7-4c31-96ae-fda5152e77e9 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:24.157876 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1280/2584] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:53:24 2022] PUT /identity/v3/projects/a24b249eaef84f608e2f2bffe2452728/users/e960e6b09a7947e682ee7c7aab06d669/roles/73e457231a824ea98c28d22610be640c => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:53:24.161909 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd0e0a5-fc10-43f0-81ba-d1fceed5c692 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:24.172822 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd0e0a5-fc10-43f0-81ba-d1fceed5c692 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:24.232472 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4709fe-c34a-4a16-a35a-9e1037c4cec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:24.233234 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4709fe-c34a-4a16-a35a-9e1037c4cec8 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:24.233521 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4709fe-c34a-4a16-a35a-9e1037c4cec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:24.233801 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4709fe-c34a-4a16-a35a-9e1037c4cec8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:24.234508 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4709fe-c34a-4a16-a35a-9e1037c4cec8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:24.236173 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4709fe-c34a-4a16-a35a-9e1037c4cec8 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:24.247237 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1305/2585] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:53:24 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 17:53:24.251126 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aca046a-d7a7-4c31-96ae-fda5152e77e9 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:24.262068 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aca046a-d7a7-4c31-96ae-fda5152e77e9 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:24.266890 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a660907-ae9b-46f6-aa77-044cafff5f79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:24.267562 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a660907-ae9b-46f6-aa77-044cafff5f79 admin admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:24.267841 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a660907-ae9b-46f6-aa77-044cafff5f79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:24.268114 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a660907-ae9b-46f6-aa77-044cafff5f79 admin admin] PATH_INFO: `/v3/projects/a24b249eaef84f608e2f2bffe2452728/users/e960e6b09a7947e682ee7c7aab06d669/roles/73e457231a824ea98c28d22610be640c` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:24.270110 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a660907-ae9b-46f6-aa77-044cafff5f79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a24b249eaef84f608e2f2bffe2452728, user_id=e960e6b09a7947e682ee7c7aab06d669, role_id=73e457231a824ea98c28d22610be640c)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:24.272732 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a660907-ae9b-46f6-aa77-044cafff5f79 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:24.284538 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1281/2586] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:53:24 2022] PUT /identity/v3/projects/a24b249eaef84f608e2f2bffe2452728/users/e960e6b09a7947e682ee7c7aab06d669/roles/73e457231a824ea98c28d22610be640c => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 17:53:24.295566 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042e428b-fdd8-4871-a5dc-0e54f978cf28 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:24.295858 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042e428b-fdd8-4871-a5dc-0e54f978cf28 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:24.296134 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042e428b-fdd8-4871-a5dc-0e54f978cf28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:24.324081 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-042e428b-fdd8-4871-a5dc-0e54f978cf28 None None] MFA Rules not processed for user `e960e6b09a7947e682ee7c7aab06d669`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:53:24.325382 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-042e428b-fdd8-4871-a5dc-0e54f978cf28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:24.369820 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:24.398665 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1306/2587] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:53:24 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:53:24.410889 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a660907-ae9b-46f6-aa77-044cafff5f79 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:24.412553 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-7a660907-ae9b-46f6-aa77-044cafff5f79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:24.446115 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:24.479485 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a660907-ae9b-46f6-aa77-044cafff5f79 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:24.483181 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763c152b-d955-452c-ad90-d8a22323af5d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96d512d0c84240918dae6c4a3e8b7b79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:24.483801 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763c152b-d955-452c-ad90-d8a22323af5d service cinder] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:24.484080 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763c152b-d955-452c-ad90-d8a22323af5d service cinder] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:24.484354 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763c152b-d955-452c-ad90-d8a22323af5d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:24.493541 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763c152b-d955-452c-ad90-d8a22323af5d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:24.495105 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763c152b-d955-452c-ad90-d8a22323af5d service cinder] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:24.504917 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1282/2588] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:53:24 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:53:24.572122 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042e428b-fdd8-4871-a5dc-0e54f978cf28 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:24.634707 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042e428b-fdd8-4871-a5dc-0e54f978cf28 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:24.640202 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa4438b-6732-43df-ab3d-d594b053611c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:24.640860 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa4438b-6732-43df-ab3d-d594b053611c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:24.641134 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa4438b-6732-43df-ab3d-d594b053611c admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:24.641434 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa4438b-6732-43df-ab3d-d594b053611c admin admin] PATH_INFO: `/v3/users/d77724fb74a74918a275f81c75ad0ea1` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:24.642590 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa4438b-6732-43df-ab3d-d594b053611c admin admin] RBAC: Authorizing `identity:delete_user(user_id=d77724fb74a74918a275f81c75ad0ea1)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:24.644493 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa4438b-6732-43df-ab3d-d594b053611c admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:24.697800 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-6aa4438b-6732-43df-ab3d-d594b053611c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd77724fb74a74918a275f81c75ad0ea1'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:53:24.704834 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-6aa4438b-6732-43df-ab3d-d594b053611c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd77724fb74a74918a275f81c75ad0ea1'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:53:24.705537 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-6aa4438b-6732-43df-ab3d-d594b053611c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd77724fb74a74918a275f81c75ad0ea1'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:53:24.712692 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-6aa4438b-6732-43df-ab3d-d594b053611c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd77724fb74a74918a275f81c75ad0ea1'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:53:24.725192 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-6aa4438b-6732-43df-ab3d-d594b053611c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd77724fb74a74918a275f81c75ad0ea1'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:53:24.731080 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1307/2589] 10.222.0.89 () {58 vars in 1224 bytes} [Thu Jul 28 17:53:24 2022] DELETE /identity/v3/users/d77724fb74a74918a275f81c75ad0ea1 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 17:53:24.735686 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fadd9a9-3533-4894-9843-798229f0f581 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:24.736172 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fadd9a9-3533-4894-9843-798229f0f581 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:24.736273 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fadd9a9-3533-4894-9843-798229f0f581 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:24.761122 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-6fadd9a9-3533-4894-9843-798229f0f581 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:53:24.763385 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-6fadd9a9-3533-4894-9843-798229f0f581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:24.829869 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1283/2590] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:53:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:53:24.840768 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa4438b-6732-43df-ab3d-d594b053611c admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:24.842312 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-6aa4438b-6732-43df-ab3d-d594b053611c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:24.901368 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa4438b-6732-43df-ab3d-d594b053611c admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:24.905353 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df9b6e7-5f1f-4eae-9601-7ad342124205 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76817cd541294331b0023069092e8e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:24.905968 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df9b6e7-5f1f-4eae-9601-7ad342124205 service neutron] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:24.906241 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df9b6e7-5f1f-4eae-9601-7ad342124205 service neutron] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:24.906514 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df9b6e7-5f1f-4eae-9601-7ad342124205 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:24.915839 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df9b6e7-5f1f-4eae-9601-7ad342124205 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:24.917333 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df9b6e7-5f1f-4eae-9601-7ad342124205 service neutron] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:24.922298 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:24.928838 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1308/2591] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:53:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:53:25.213816 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fadd9a9-3533-4894-9843-798229f0f581 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:25.215239 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-6fadd9a9-3533-4894-9843-798229f0f581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:25.230605 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fadd9a9-3533-4894-9843-798229f0f581 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:25.236262 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd1eca6-fb21-439f-bab5-bb6d3adc331a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:25.237268 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd1eca6-fb21-439f-bab5-bb6d3adc331a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:25.237546 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd1eca6-fb21-439f-bab5-bb6d3adc331a admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:25.237823 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd1eca6-fb21-439f-bab5-bb6d3adc331a admin admin] PATH_INFO: `/v3/projects/814856f317024d35ae0f46fce85a7ec3` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:25.238995 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd1eca6-fb21-439f-bab5-bb6d3adc331a admin admin] RBAC: Authorizing `identity:delete_project(project_id=814856f317024d35ae0f46fce85a7ec3)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:25.240852 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd1eca6-fb21-439f-bab5-bb6d3adc331a admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:25.291973 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-6bd1eca6-fb21-439f-bab5-bb6d3adc331a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '814856f317024d35ae0f46fce85a7ec3', 'request_id': 'req-6bd1eca6-fb21-439f-bab5-bb6d3adc331a'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:53:25.298781 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-6bd1eca6-fb21-439f-bab5-bb6d3adc331a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '814856f317024d35ae0f46fce85a7ec3', 'request_id': 'req-6bd1eca6-fb21-439f-bab5-bb6d3adc331a'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:53:25.304641 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-6bd1eca6-fb21-439f-bab5-bb6d3adc331a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '814856f317024d35ae0f46fce85a7ec3', 'request_id': 'req-6bd1eca6-fb21-439f-bab5-bb6d3adc331a'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:53:25.320118 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-6bd1eca6-fb21-439f-bab5-bb6d3adc331a admin admin] The token cache is being invalidate because project 814856f317024d35ae0f46fce85a7ec3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80325) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 17:53:25.320418 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-6bd1eca6-fb21-439f-bab5-bb6d3adc331a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:53:25.321150 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-6bd1eca6-fb21-439f-bab5-bb6d3adc331a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:53:25.323022 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1284/2592] 10.222.0.89 () {58 vars in 1233 bytes} [Thu Jul 28 17:53:25 2022] DELETE /identity/v3/projects/814856f317024d35ae0f46fce85a7ec3 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:53:25.327066 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df9b6e7-5f1f-4eae-9601-7ad342124205 service neutron] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:25.337096 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df9b6e7-5f1f-4eae-9601-7ad342124205 service neutron] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:25.397971 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fead335f-b90a-4ed9-9d40-c820634160e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:25.398612 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fead335f-b90a-4ed9-9d40-c820634160e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:25.398893 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fead335f-b90a-4ed9-9d40-c820634160e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:25.399164 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fead335f-b90a-4ed9-9d40-c820634160e8 admin admin] PATH_INFO: `/v3/users/e960e6b09a7947e682ee7c7aab06d669` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:25.400348 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fead335f-b90a-4ed9-9d40-c820634160e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e960e6b09a7947e682ee7c7aab06d669)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:25.402777 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fead335f-b90a-4ed9-9d40-c820634160e8 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:25.456507 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-fead335f-b90a-4ed9-9d40-c820634160e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e960e6b09a7947e682ee7c7aab06d669'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:53:25.470118 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-fead335f-b90a-4ed9-9d40-c820634160e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e960e6b09a7947e682ee7c7aab06d669'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:53:25.470951 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-fead335f-b90a-4ed9-9d40-c820634160e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e960e6b09a7947e682ee7c7aab06d669'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:53:25.495174 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-fead335f-b90a-4ed9-9d40-c820634160e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e960e6b09a7947e682ee7c7aab06d669'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:53:25.509653 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-fead335f-b90a-4ed9-9d40-c820634160e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e960e6b09a7947e682ee7c7aab06d669'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:53:25.516649 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1309/2593] 10.222.0.89 () {58 vars in 1224 bytes} [Thu Jul 28 17:53:25 2022] DELETE /identity/v3/users/e960e6b09a7947e682ee7c7aab06d669 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 17:53:25.812914 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd1eca6-fb21-439f-bab5-bb6d3adc331a admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:25.814535 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-6bd1eca6-fb21-439f-bab5-bb6d3adc331a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:25.827961 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd1eca6-fb21-439f-bab5-bb6d3adc331a admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:25.879798 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:25.893379 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f235362-7dc5-4c26-8574-488052ad8ec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:25.894031 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f235362-7dc5-4c26-8574-488052ad8ec7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:25.894312 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f235362-7dc5-4c26-8574-488052ad8ec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:25.894584 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f235362-7dc5-4c26-8574-488052ad8ec7 admin admin] PATH_INFO: `/v3/projects/a24b249eaef84f608e2f2bffe2452728` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:25.895791 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f235362-7dc5-4c26-8574-488052ad8ec7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a24b249eaef84f608e2f2bffe2452728)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:25.897747 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f235362-7dc5-4c26-8574-488052ad8ec7 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:25.958537 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-7f235362-7dc5-4c26-8574-488052ad8ec7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a24b249eaef84f608e2f2bffe2452728', 'request_id': 'req-7f235362-7dc5-4c26-8574-488052ad8ec7'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:53:25.965863 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-7f235362-7dc5-4c26-8574-488052ad8ec7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a24b249eaef84f608e2f2bffe2452728', 'request_id': 'req-7f235362-7dc5-4c26-8574-488052ad8ec7'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:53:25.972493 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-7f235362-7dc5-4c26-8574-488052ad8ec7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a24b249eaef84f608e2f2bffe2452728', 'request_id': 'req-7f235362-7dc5-4c26-8574-488052ad8ec7'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:53:25.987797 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-7f235362-7dc5-4c26-8574-488052ad8ec7 admin admin] The token cache is being invalidate because project a24b249eaef84f608e2f2bffe2452728 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80325) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 17:53:25.988093 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-7f235362-7dc5-4c26-8574-488052ad8ec7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:53:25.988713 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-7f235362-7dc5-4c26-8574-488052ad8ec7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:53:25.990258 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1285/2594] 10.222.0.89 () {58 vars in 1233 bytes} [Thu Jul 28 17:53:25 2022] DELETE /identity/v3/projects/a24b249eaef84f608e2f2bffe2452728 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 17:53:25.996924 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec90914-7ad4-4f38-a60d-0f20a4efc2c9 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:25.997482 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec90914-7ad4-4f38-a60d-0f20a4efc2c9 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:25.997661 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec90914-7ad4-4f38-a60d-0f20a4efc2c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:26.018932 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-dec90914-7ad4-4f38-a60d-0f20a4efc2c9 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:53:26.020183 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-dec90914-7ad4-4f38-a60d-0f20a4efc2c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:26.081453 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1310/2595] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:53:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:53:26.090418 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5748642-f9a3-4264-8cbe-05d0033955c2 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:26.090698 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5748642-f9a3-4264-8cbe-05d0033955c2 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:26.090971 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5748642-f9a3-4264-8cbe-05d0033955c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:26.112319 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-e5748642-f9a3-4264-8cbe-05d0033955c2 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:53:26.113558 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-e5748642-f9a3-4264-8cbe-05d0033955c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:26.122453 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1286/2596] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:53:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:53:26.126788 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec90914-7ad4-4f38-a60d-0f20a4efc2c9 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:26.137508 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec90914-7ad4-4f38-a60d-0f20a4efc2c9 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:26.142250 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a08262-4a8e-4ba1-83fd-0d4af693f6d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:26.143268 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a08262-4a8e-4ba1-83fd-0d4af693f6d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:26.143268 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a08262-4a8e-4ba1-83fd-0d4af693f6d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:26.143564 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a08262-4a8e-4ba1-83fd-0d4af693f6d1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:26.144080 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a08262-4a8e-4ba1-83fd-0d4af693f6d1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:26.144387 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a08262-4a8e-4ba1-83fd-0d4af693f6d1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:26.146034 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a08262-4a8e-4ba1-83fd-0d4af693f6d1 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:26.162731 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1311/2597] 10.222.0.89 () {58 vars in 1153 bytes} [Thu Jul 28 17:53:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:26.163887 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5748642-f9a3-4264-8cbe-05d0033955c2 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:26.173061 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5748642-f9a3-4264-8cbe-05d0033955c2 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:26.177783 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c23509-cae7-4193-bc1e-2c55d18d34d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:26.178385 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c23509-cae7-4193-bc1e-2c55d18d34d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:26.180060 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c23509-cae7-4193-bc1e-2c55d18d34d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:26.180060 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c23509-cae7-4193-bc1e-2c55d18d34d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:26.180060 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c23509-cae7-4193-bc1e-2c55d18d34d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:26.183030 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c23509-cae7-4193-bc1e-2c55d18d34d8 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:26.214219 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1287/2598] 10.222.0.89 () {60 vars in 1153 bytes} [Thu Jul 28 17:53:26 2022] POST /identity/v3/projects => generated 407 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 17:53:26.217921 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a08262-4a8e-4ba1-83fd-0d4af693f6d1 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:26.229901 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a08262-4a8e-4ba1-83fd-0d4af693f6d1 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:26.234677 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:26.299672 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703cd296-3f70-401f-a152-6dab0b4c904b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:26.300295 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703cd296-3f70-401f-a152-6dab0b4c904b admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:26.300573 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703cd296-3f70-401f-a152-6dab0b4c904b admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:26.300841 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703cd296-3f70-401f-a152-6dab0b4c904b admin admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:26.301581 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703cd296-3f70-401f-a152-6dab0b4c904b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:26.303499 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703cd296-3f70-401f-a152-6dab0b4c904b admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:26.330154 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1312/2599] 10.222.0.89 () {60 vars in 1144 bytes} [Thu Jul 28 17:53:26 2022] POST /identity/v3/users => generated 304 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 17:53:26.340255 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c23509-cae7-4193-bc1e-2c55d18d34d8 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:26.352627 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c23509-cae7-4193-bc1e-2c55d18d34d8 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:26.358958 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0861084f-50a7-4944-841a-52872d1f5f62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:26.359571 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0861084f-50a7-4944-841a-52872d1f5f62 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:26.359873 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0861084f-50a7-4944-841a-52872d1f5f62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:26.360137 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0861084f-50a7-4944-841a-52872d1f5f62 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:26.360987 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0861084f-50a7-4944-841a-52872d1f5f62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:26.362606 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0861084f-50a7-4944-841a-52872d1f5f62 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:26.375062 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1288/2600] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:53:26 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:53:26.379770 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703cd296-3f70-401f-a152-6dab0b4c904b admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:26.390652 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703cd296-3f70-401f-a152-6dab0b4c904b admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:26.395886 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685a052c-d91d-42db-b6e9-88b5a5f08075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:26.396400 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685a052c-d91d-42db-b6e9-88b5a5f08075 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:26.396704 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685a052c-d91d-42db-b6e9-88b5a5f08075 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:26.397077 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685a052c-d91d-42db-b6e9-88b5a5f08075 admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:26.397939 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685a052c-d91d-42db-b6e9-88b5a5f08075 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:26.399558 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685a052c-d91d-42db-b6e9-88b5a5f08075 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:26.412231 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1313/2601] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:53:26 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:53:26.417291 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0861084f-50a7-4944-841a-52872d1f5f62 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:26.428806 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0861084f-50a7-4944-841a-52872d1f5f62 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:26.434995 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7841cd-42ab-47c8-bce3-c4f6a1e1c9f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:26.435835 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7841cd-42ab-47c8-bce3-c4f6a1e1c9f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:26.436240 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7841cd-42ab-47c8-bce3-c4f6a1e1c9f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:26.436577 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7841cd-42ab-47c8-bce3-c4f6a1e1c9f3 admin admin] PATH_INFO: `/v3/projects/c216496b297e4ad5875a43450574fa45/users/ad31b80430b14884b5f389a45d86e2b3/roles/11f8731756fe4caca13efa61b71b11c5` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:26.455667 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7841cd-42ab-47c8-bce3-c4f6a1e1c9f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c216496b297e4ad5875a43450574fa45, user_id=ad31b80430b14884b5f389a45d86e2b3, role_id=11f8731756fe4caca13efa61b71b11c5)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:26.460012 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7841cd-42ab-47c8-bce3-c4f6a1e1c9f3 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:26.495441 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1289/2602] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:53:26 2022] PUT /identity/v3/projects/c216496b297e4ad5875a43450574fa45/users/ad31b80430b14884b5f389a45d86e2b3/roles/11f8731756fe4caca13efa61b71b11c5 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:53:26.506027 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2922ca3-f6e7-49c7-8a71-8c703b065c44 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:26.506765 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2922ca3-f6e7-49c7-8a71-8c703b065c44 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:26.507361 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2922ca3-f6e7-49c7-8a71-8c703b065c44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:26.574648 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-e2922ca3-f6e7-49c7-8a71-8c703b065c44 None None] MFA Rules not processed for user `ad31b80430b14884b5f389a45d86e2b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:53:26.576347 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-e2922ca3-f6e7-49c7-8a71-8c703b065c44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:26.672139 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1314/2603] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:53:26 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:53:26.676913 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7841cd-42ab-47c8-bce3-c4f6a1e1c9f3 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:26.743095 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7841cd-42ab-47c8-bce3-c4f6a1e1c9f3 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:26.748467 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ad7032-5ff6-4f32-9e64-b4649bcd07cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:26.749084 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ad7032-5ff6-4f32-9e64-b4649bcd07cc admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:26.749393 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ad7032-5ff6-4f32-9e64-b4649bcd07cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:26.749665 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ad7032-5ff6-4f32-9e64-b4649bcd07cc admin admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:26.750359 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ad7032-5ff6-4f32-9e64-b4649bcd07cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:26.752245 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ad7032-5ff6-4f32-9e64-b4649bcd07cc admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:26.781365 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1290/2604] 10.222.0.89 () {60 vars in 1153 bytes} [Thu Jul 28 17:53:26 2022] POST /identity/v3/projects => generated 407 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:53:26.785705 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2922ca3-f6e7-49c7-8a71-8c703b065c44 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:26.798251 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2922ca3-f6e7-49c7-8a71-8c703b065c44 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:26.875020 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0abff1-c49f-4c6a-b9e6-b832dec4f082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:26.875666 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0abff1-c49f-4c6a-b9e6-b832dec4f082 admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:26.875950 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0abff1-c49f-4c6a-b9e6-b832dec4f082 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:26.876226 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0abff1-c49f-4c6a-b9e6-b832dec4f082 admin admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:26.876918 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0abff1-c49f-4c6a-b9e6-b832dec4f082 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:26.879100 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0abff1-c49f-4c6a-b9e6-b832dec4f082 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:26.908867 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1315/2605] 10.222.0.89 () {60 vars in 1144 bytes} [Thu Jul 28 17:53:26 2022] POST /identity/v3/users => generated 310 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:53:26.910170 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ad7032-5ff6-4f32-9e64-b4649bcd07cc admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:26.916837 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:26.925454 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ad7032-5ff6-4f32-9e64-b4649bcd07cc admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:26.933953 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38566aa5-c8b5-4d4e-8470-633e522f71b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:26.934605 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38566aa5-c8b5-4d4e-8470-633e522f71b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:26.935001 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38566aa5-c8b5-4d4e-8470-633e522f71b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:26.935291 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38566aa5-c8b5-4d4e-8470-633e522f71b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:26.936002 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38566aa5-c8b5-4d4e-8470-633e522f71b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:26.937738 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38566aa5-c8b5-4d4e-8470-633e522f71b0 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:26.950461 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1291/2606] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:53:26 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 17:53:26.954469 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0abff1-c49f-4c6a-b9e6-b832dec4f082 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:26.966115 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:26.973064 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0abff1-c49f-4c6a-b9e6-b832dec4f082 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:26.979908 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8403bab3-1665-44e9-8738-b8aba0f4f9d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:26.980533 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8403bab3-1665-44e9-8738-b8aba0f4f9d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:26.980821 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8403bab3-1665-44e9-8738-b8aba0f4f9d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:26.981217 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8403bab3-1665-44e9-8738-b8aba0f4f9d1 admin admin] PATH_INFO: `/v3/projects/08bde4c16d254e27a89d05ec3276c7af/users/a8044308dc5946719ff54a8bb022605b/roles/73e457231a824ea98c28d22610be640c` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:27.003380 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8403bab3-1665-44e9-8738-b8aba0f4f9d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08bde4c16d254e27a89d05ec3276c7af, user_id=a8044308dc5946719ff54a8bb022605b, role_id=73e457231a824ea98c28d22610be640c)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:27.007609 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8403bab3-1665-44e9-8738-b8aba0f4f9d1 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:27.025228 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1316/2607] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:53:26 2022] PUT /identity/v3/projects/08bde4c16d254e27a89d05ec3276c7af/users/a8044308dc5946719ff54a8bb022605b/roles/73e457231a824ea98c28d22610be640c => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:53:27.027295 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38566aa5-c8b5-4d4e-8470-633e522f71b0 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:27.039569 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38566aa5-c8b5-4d4e-8470-633e522f71b0 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:27.121914 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1df92c-9e93-4017-bf56-7f5c846ce00a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:27.122817 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1df92c-9e93-4017-bf56-7f5c846ce00a admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:27.123171 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1df92c-9e93-4017-bf56-7f5c846ce00a admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:27.123516 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1df92c-9e93-4017-bf56-7f5c846ce00a admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:27.124326 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1df92c-9e93-4017-bf56-7f5c846ce00a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:27.126094 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1df92c-9e93-4017-bf56-7f5c846ce00a admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:27.138240 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1292/2608] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:53:27 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:53:27.142578 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8403bab3-1665-44e9-8738-b8aba0f4f9d1 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:27.155553 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8403bab3-1665-44e9-8738-b8aba0f4f9d1 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:27.161044 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1d9950-9964-42d8-8082-ef7f3e66cc26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:27.161850 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1d9950-9964-42d8-8082-ef7f3e66cc26 admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:27.162315 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1d9950-9964-42d8-8082-ef7f3e66cc26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:27.162600 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1d9950-9964-42d8-8082-ef7f3e66cc26 admin admin] PATH_INFO: `/v3/projects/08bde4c16d254e27a89d05ec3276c7af/users/a8044308dc5946719ff54a8bb022605b/roles/73e457231a824ea98c28d22610be640c` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:27.164788 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1d9950-9964-42d8-8082-ef7f3e66cc26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08bde4c16d254e27a89d05ec3276c7af, user_id=a8044308dc5946719ff54a8bb022605b, role_id=73e457231a824ea98c28d22610be640c)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:27.168517 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1d9950-9964-42d8-8082-ef7f3e66cc26 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:27.187441 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1317/2609] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:53:27 2022] PUT /identity/v3/projects/08bde4c16d254e27a89d05ec3276c7af/users/a8044308dc5946719ff54a8bb022605b/roles/73e457231a824ea98c28d22610be640c => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 17:53:27.192720 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9976d8-6bd3-4232-aed6-a1d0ef0a33e8 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:27.193124 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9976d8-6bd3-4232-aed6-a1d0ef0a33e8 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:27.193491 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9976d8-6bd3-4232-aed6-a1d0ef0a33e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:27.223653 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-bb9976d8-6bd3-4232-aed6-a1d0ef0a33e8 None None] MFA Rules not processed for user `a8044308dc5946719ff54a8bb022605b`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:53:27.225031 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-bb9976d8-6bd3-4232-aed6-a1d0ef0a33e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:27.308157 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1293/2610] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:53:27 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:53:27.320368 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1d9950-9964-42d8-8082-ef7f3e66cc26 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:27.322147 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-cf1d9950-9964-42d8-8082-ef7f3e66cc26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:27.382950 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1d9950-9964-42d8-8082-ef7f3e66cc26 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:27.387494 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7795ead0-2ee5-4f6d-b537-44e08ecf0f98 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96d512d0c84240918dae6c4a3e8b7b79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:27.388117 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7795ead0-2ee5-4f6d-b537-44e08ecf0f98 service cinder] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:27.388406 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7795ead0-2ee5-4f6d-b537-44e08ecf0f98 service cinder] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:27.388675 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7795ead0-2ee5-4f6d-b537-44e08ecf0f98 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:27.398056 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7795ead0-2ee5-4f6d-b537-44e08ecf0f98 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:27.399500 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7795ead0-2ee5-4f6d-b537-44e08ecf0f98 service cinder] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:27.410720 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1318/2611] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:53:27 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:53:27.571584 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9976d8-6bd3-4232-aed6-a1d0ef0a33e8 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:27.584036 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9976d8-6bd3-4232-aed6-a1d0ef0a33e8 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:27.589211 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b3b7c6-4ede-47b7-99fe-ce44d85f1955 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96d512d0c84240918dae6c4a3e8b7b79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:27.589483 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b3b7c6-4ede-47b7-99fe-ce44d85f1955 service cinder] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:27.590406 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b3b7c6-4ede-47b7-99fe-ce44d85f1955 service cinder] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:27.590785 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b3b7c6-4ede-47b7-99fe-ce44d85f1955 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:27.675175 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b3b7c6-4ede-47b7-99fe-ce44d85f1955 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:27.677413 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b3b7c6-4ede-47b7-99fe-ce44d85f1955 service cinder] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:27.691522 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1294/2612] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:53:27 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:53:30.849266 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7370ed8a-400e-4782-bece-4dbc77a3c08c None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:30.849755 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7370ed8a-400e-4782-bece-4dbc77a3c08c None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:30.849888 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7370ed8a-400e-4782-bece-4dbc77a3c08c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:30.871394 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-7370ed8a-400e-4782-bece-4dbc77a3c08c None None] MFA Rules not processed for user `39ff02cba2fd4e7b9fb27b98b9daa82e`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:53:30.872630 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-7370ed8a-400e-4782-bece-4dbc77a3c08c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:30.931138 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1319/2613] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:53:30 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:53:35.718382 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecf3fb8-abdf-4e39-9fb8-a65c3be6081e None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:35.718906 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecf3fb8-abdf-4e39-9fb8-a65c3be6081e None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:35.718906 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecf3fb8-abdf-4e39-9fb8-a65c3be6081e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:35.741206 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=100 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:35.754608 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-8ecf3fb8-abdf-4e39-9fb8-a65c3be6081e None None] MFA Rules not processed for user `c44387d13a694ee6bb47a13906003330`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:53:35.755698 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-8ecf3fb8-abdf-4e39-9fb8-a65c3be6081e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:35.831536 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1295/2614] 10.222.0.89 () {58 vars in 1009 bytes} [Thu Jul 28 17:53:35 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:53:35.895783 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7370ed8a-400e-4782-bece-4dbc77a3c08c None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:35.897495 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-7370ed8a-400e-4782-bece-4dbc77a3c08c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:36.037778 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7370ed8a-400e-4782-bece-4dbc77a3c08c None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:36.047518 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceda15b5-0712-48d7-ac02-91d2f06f1b32 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c59473b01bf4433ba3b111114fc048b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:36.049437 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceda15b5-0712-48d7-ac02-91d2f06f1b32 service swift] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:36.050429 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceda15b5-0712-48d7-ac02-91d2f06f1b32 service swift] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:36.051369 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceda15b5-0712-48d7-ac02-91d2f06f1b32 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:36.081614 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceda15b5-0712-48d7-ac02-91d2f06f1b32 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:36.083236 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceda15b5-0712-48d7-ac02-91d2f06f1b32 service swift] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:36.099954 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1320/2615] 10.222.0.89 () {60 vars in 1384 bytes} [Thu Jul 28 17:53:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 17:53:38.447931 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecf3fb8-abdf-4e39-9fb8-a65c3be6081e None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:38.536355 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecf3fb8-abdf-4e39-9fb8-a65c3be6081e None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:38.541316 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109e7fb1-d0b7-48e4-9b6a-3434d7ce03cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:38.541996 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109e7fb1-d0b7-48e4-9b6a-3434d7ce03cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:38.542295 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109e7fb1-d0b7-48e4-9b6a-3434d7ce03cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:38.542567 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109e7fb1-d0b7-48e4-9b6a-3434d7ce03cd admin admin] PATH_INFO: `/v3/users/ad31b80430b14884b5f389a45d86e2b3` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:38.543801 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109e7fb1-d0b7-48e4-9b6a-3434d7ce03cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad31b80430b14884b5f389a45d86e2b3)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:38.545852 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109e7fb1-d0b7-48e4-9b6a-3434d7ce03cd admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:38.612798 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-109e7fb1-d0b7-48e4-9b6a-3434d7ce03cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad31b80430b14884b5f389a45d86e2b3'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:53:38.624236 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-109e7fb1-d0b7-48e4-9b6a-3434d7ce03cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad31b80430b14884b5f389a45d86e2b3'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:53:38.645894 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-109e7fb1-d0b7-48e4-9b6a-3434d7ce03cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad31b80430b14884b5f389a45d86e2b3'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:53:38.647406 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-109e7fb1-d0b7-48e4-9b6a-3434d7ce03cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad31b80430b14884b5f389a45d86e2b3'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:53:38.648934 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-109e7fb1-d0b7-48e4-9b6a-3434d7ce03cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad31b80430b14884b5f389a45d86e2b3'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:53:38.673095 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1296/2616] 10.222.0.89 () {58 vars in 1224 bytes} [Thu Jul 28 17:53:38 2022] DELETE /identity/v3/users/ad31b80430b14884b5f389a45d86e2b3 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:53:38.678034 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5cc501-56d1-4e46-ba41-d09fcb9bbd31 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:38.678561 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5cc501-56d1-4e46-ba41-d09fcb9bbd31 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:38.678803 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5cc501-56d1-4e46-ba41-d09fcb9bbd31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:38.688530 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:38.721822 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-cc5cc501-56d1-4e46-ba41-d09fcb9bbd31 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:53:38.724397 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-cc5cc501-56d1-4e46-ba41-d09fcb9bbd31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:38.800710 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1321/2617] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:53:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:53:38.807516 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109e7fb1-d0b7-48e4-9b6a-3434d7ce03cd admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:38.809201 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-109e7fb1-d0b7-48e4-9b6a-3434d7ce03cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:38.855453 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:38.890129 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109e7fb1-d0b7-48e4-9b6a-3434d7ce03cd admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:38.894221 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1118c600-3419-44cd-867a-2280cb91b9a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76817cd541294331b0023069092e8e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:38.894834 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1118c600-3419-44cd-867a-2280cb91b9a1 service neutron] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:38.895114 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1118c600-3419-44cd-867a-2280cb91b9a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:38.895390 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1118c600-3419-44cd-867a-2280cb91b9a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:38.909286 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1118c600-3419-44cd-867a-2280cb91b9a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:38.911261 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1118c600-3419-44cd-867a-2280cb91b9a1 service neutron] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:38.923924 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1297/2618] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:53:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:53:39.346438 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5cc501-56d1-4e46-ba41-d09fcb9bbd31 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:39.348033 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-cc5cc501-56d1-4e46-ba41-d09fcb9bbd31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:39.364386 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5cc501-56d1-4e46-ba41-d09fcb9bbd31 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:39.369827 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bc4062-779b-4414-9709-ed6e2159bf22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:39.370807 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bc4062-779b-4414-9709-ed6e2159bf22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:39.371150 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bc4062-779b-4414-9709-ed6e2159bf22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:39.371358 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bc4062-779b-4414-9709-ed6e2159bf22 admin admin] PATH_INFO: `/v3/projects/c216496b297e4ad5875a43450574fa45` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:39.372821 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bc4062-779b-4414-9709-ed6e2159bf22 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c216496b297e4ad5875a43450574fa45)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:39.375623 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bc4062-779b-4414-9709-ed6e2159bf22 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:39.431832 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-48bc4062-779b-4414-9709-ed6e2159bf22 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c216496b297e4ad5875a43450574fa45', 'request_id': 'req-48bc4062-779b-4414-9709-ed6e2159bf22'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:53:39.448591 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-48bc4062-779b-4414-9709-ed6e2159bf22 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c216496b297e4ad5875a43450574fa45', 'request_id': 'req-48bc4062-779b-4414-9709-ed6e2159bf22'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:53:39.456407 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-48bc4062-779b-4414-9709-ed6e2159bf22 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c216496b297e4ad5875a43450574fa45', 'request_id': 'req-48bc4062-779b-4414-9709-ed6e2159bf22'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:53:39.473579 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-48bc4062-779b-4414-9709-ed6e2159bf22 admin admin] The token cache is being invalidate because project c216496b297e4ad5875a43450574fa45 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80324) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 17:53:39.474023 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-48bc4062-779b-4414-9709-ed6e2159bf22 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:53:39.474745 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-48bc4062-779b-4414-9709-ed6e2159bf22 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:53:39.477209 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1322/2619] 10.222.0.89 () {58 vars in 1233 bytes} [Thu Jul 28 17:53:39 2022] DELETE /identity/v3/projects/c216496b297e4ad5875a43450574fa45 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:53:39.482005 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1118c600-3419-44cd-867a-2280cb91b9a1 service neutron] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:39.492134 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1118c600-3419-44cd-867a-2280cb91b9a1 service neutron] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:39.558495 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf92e23-5fa5-4579-8068-a687500d0296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:39.559269 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf92e23-5fa5-4579-8068-a687500d0296 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:39.559617 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf92e23-5fa5-4579-8068-a687500d0296 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:39.560034 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf92e23-5fa5-4579-8068-a687500d0296 admin admin] PATH_INFO: `/v3/users/a8044308dc5946719ff54a8bb022605b` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:39.561363 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf92e23-5fa5-4579-8068-a687500d0296 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8044308dc5946719ff54a8bb022605b)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:39.563356 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf92e23-5fa5-4579-8068-a687500d0296 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:39.649513 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-cbf92e23-5fa5-4579-8068-a687500d0296 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8044308dc5946719ff54a8bb022605b'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:53:39.663522 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-cbf92e23-5fa5-4579-8068-a687500d0296 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8044308dc5946719ff54a8bb022605b'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:53:39.677072 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-cbf92e23-5fa5-4579-8068-a687500d0296 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8044308dc5946719ff54a8bb022605b'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:53:39.687722 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-cbf92e23-5fa5-4579-8068-a687500d0296 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8044308dc5946719ff54a8bb022605b'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:53:39.688695 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-cbf92e23-5fa5-4579-8068-a687500d0296 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8044308dc5946719ff54a8bb022605b'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:53:39.738409 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1298/2620] 10.222.0.89 () {58 vars in 1224 bytes} [Thu Jul 28 17:53:39 2022] DELETE /identity/v3/users/a8044308dc5946719ff54a8bb022605b => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:53:40.248619 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bc4062-779b-4414-9709-ed6e2159bf22 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:40.250226 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-48bc4062-779b-4414-9709-ed6e2159bf22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:40.269293 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bc4062-779b-4414-9709-ed6e2159bf22 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:40.346935 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958bdc50-13d5-4a0d-bd1f-98b0d5780221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:40.348089 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958bdc50-13d5-4a0d-bd1f-98b0d5780221 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:40.348568 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958bdc50-13d5-4a0d-bd1f-98b0d5780221 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:40.349185 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958bdc50-13d5-4a0d-bd1f-98b0d5780221 admin admin] PATH_INFO: `/v3/projects/08bde4c16d254e27a89d05ec3276c7af` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:40.351436 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958bdc50-13d5-4a0d-bd1f-98b0d5780221 admin admin] RBAC: Authorizing `identity:delete_project(project_id=08bde4c16d254e27a89d05ec3276c7af)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:40.355223 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958bdc50-13d5-4a0d-bd1f-98b0d5780221 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:40.398243 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:40.418104 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-958bdc50-13d5-4a0d-bd1f-98b0d5780221 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08bde4c16d254e27a89d05ec3276c7af', 'request_id': 'req-958bdc50-13d5-4a0d-bd1f-98b0d5780221'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:53:40.426050 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-958bdc50-13d5-4a0d-bd1f-98b0d5780221 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08bde4c16d254e27a89d05ec3276c7af', 'request_id': 'req-958bdc50-13d5-4a0d-bd1f-98b0d5780221'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:53:40.433192 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-958bdc50-13d5-4a0d-bd1f-98b0d5780221 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08bde4c16d254e27a89d05ec3276c7af', 'request_id': 'req-958bdc50-13d5-4a0d-bd1f-98b0d5780221'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:53:40.449185 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-958bdc50-13d5-4a0d-bd1f-98b0d5780221 admin admin] The token cache is being invalidate because project 08bde4c16d254e27a89d05ec3276c7af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80324) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 17:53:40.449619 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-958bdc50-13d5-4a0d-bd1f-98b0d5780221 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:53:40.450481 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-958bdc50-13d5-4a0d-bd1f-98b0d5780221 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:53:40.452317 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1323/2621] 10.222.0.89 () {58 vars in 1233 bytes} [Thu Jul 28 17:53:40 2022] DELETE /identity/v3/projects/08bde4c16d254e27a89d05ec3276c7af => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:53:40.456912 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33f570d-09fd-43c3-8b4d-acb1bd630d9f None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:40.457367 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33f570d-09fd-43c3-8b4d-acb1bd630d9f None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:40.457651 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33f570d-09fd-43c3-8b4d-acb1bd630d9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:40.481122 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-a33f570d-09fd-43c3-8b4d-acb1bd630d9f None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:53:40.482442 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-a33f570d-09fd-43c3-8b4d-acb1bd630d9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:40.531447 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:40.550327 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1299/2622] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:53:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:53:40.559913 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870396fa-2187-4764-8b9b-2e349a3a18dd None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:40.560332 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870396fa-2187-4764-8b9b-2e349a3a18dd None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:40.560822 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870396fa-2187-4764-8b9b-2e349a3a18dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:40.591826 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-870396fa-2187-4764-8b9b-2e349a3a18dd None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:53:40.593866 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-870396fa-2187-4764-8b9b-2e349a3a18dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:40.609434 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1324/2623] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:53:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:53:40.611581 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33f570d-09fd-43c3-8b4d-acb1bd630d9f None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:40.625791 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33f570d-09fd-43c3-8b4d-acb1bd630d9f None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:40.631020 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbed479-6586-4476-bcb2-150718db33c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:40.631721 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbed479-6586-4476-bcb2-150718db33c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:40.632103 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbed479-6586-4476-bcb2-150718db33c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:40.632379 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbed479-6586-4476-bcb2-150718db33c4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:40.633302 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbed479-6586-4476-bcb2-150718db33c4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:40.633642 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbed479-6586-4476-bcb2-150718db33c4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:40.636457 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbed479-6586-4476-bcb2-150718db33c4 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:40.649427 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1300/2624] 10.222.0.89 () {58 vars in 1153 bytes} [Thu Jul 28 17:53:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 17:53:40.657906 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870396fa-2187-4764-8b9b-2e349a3a18dd None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:40.671355 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870396fa-2187-4764-8b9b-2e349a3a18dd None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:40.678386 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff84d4bc-da31-43bc-834c-32d8be77cff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:40.679373 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff84d4bc-da31-43bc-834c-32d8be77cff2 admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:40.679827 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff84d4bc-da31-43bc-834c-32d8be77cff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:40.683268 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff84d4bc-da31-43bc-834c-32d8be77cff2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:40.684103 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff84d4bc-da31-43bc-834c-32d8be77cff2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:40.686588 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff84d4bc-da31-43bc-834c-32d8be77cff2 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:40.726871 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1325/2625] 10.222.0.89 () {60 vars in 1153 bytes} [Thu Jul 28 17:53:40 2022] POST /identity/v3/projects => generated 379 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:53:40.732161 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbed479-6586-4476-bcb2-150718db33c4 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:40.748112 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbed479-6586-4476-bcb2-150718db33c4 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:40.837806 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0894329-854d-4a7a-90ea-24a5eac4f5d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:40.839506 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0894329-854d-4a7a-90ea-24a5eac4f5d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:40.840042 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0894329-854d-4a7a-90ea-24a5eac4f5d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:40.840376 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0894329-854d-4a7a-90ea-24a5eac4f5d1 admin admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:40.841321 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0894329-854d-4a7a-90ea-24a5eac4f5d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:40.843863 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0894329-854d-4a7a-90ea-24a5eac4f5d1 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:40.871097 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1301/2626] 10.222.0.89 () {60 vars in 1144 bytes} [Thu Jul 28 17:53:40 2022] POST /identity/v3/users => generated 290 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:53:40.876008 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff84d4bc-da31-43bc-834c-32d8be77cff2 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:40.894584 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff84d4bc-da31-43bc-834c-32d8be77cff2 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:40.902991 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b736dd3-d1f9-44a4-ab34-5211e6bbf9c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:40.903766 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b736dd3-d1f9-44a4-ab34-5211e6bbf9c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:40.904048 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b736dd3-d1f9-44a4-ab34-5211e6bbf9c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:40.904456 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b736dd3-d1f9-44a4-ab34-5211e6bbf9c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:40.905407 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b736dd3-d1f9-44a4-ab34-5211e6bbf9c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:40.907968 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b736dd3-d1f9-44a4-ab34-5211e6bbf9c8 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:40.936371 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1326/2627] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:53:40 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:53:40.937688 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0894329-854d-4a7a-90ea-24a5eac4f5d1 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:40.949815 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0894329-854d-4a7a-90ea-24a5eac4f5d1 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:40.955014 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4a9875-8307-4e6e-9acf-68c15a17044c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:40.955636 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4a9875-8307-4e6e-9acf-68c15a17044c admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:40.955990 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4a9875-8307-4e6e-9acf-68c15a17044c admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:40.956197 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4a9875-8307-4e6e-9acf-68c15a17044c admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:40.956899 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4a9875-8307-4e6e-9acf-68c15a17044c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:40.958560 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4a9875-8307-4e6e-9acf-68c15a17044c admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:40.978339 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1302/2628] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:53:40 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:53:40.981082 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b736dd3-d1f9-44a4-ab34-5211e6bbf9c8 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:40.996862 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b736dd3-d1f9-44a4-ab34-5211e6bbf9c8 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:41.003935 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341ac9f4-0908-4f4f-a71e-45f3fd6ba8e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:41.004767 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341ac9f4-0908-4f4f-a71e-45f3fd6ba8e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:41.005054 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341ac9f4-0908-4f4f-a71e-45f3fd6ba8e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:41.005477 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341ac9f4-0908-4f4f-a71e-45f3fd6ba8e9 admin admin] PATH_INFO: `/v3/projects/82a9c61f73594105844658e5abf067dc/users/a721daa9088941ed9a4b85a42deeab0c/roles/11f8731756fe4caca13efa61b71b11c5` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:41.030433 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341ac9f4-0908-4f4f-a71e-45f3fd6ba8e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=82a9c61f73594105844658e5abf067dc, user_id=a721daa9088941ed9a4b85a42deeab0c, role_id=11f8731756fe4caca13efa61b71b11c5)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:41.033360 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341ac9f4-0908-4f4f-a71e-45f3fd6ba8e9 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:41.053376 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1327/2629] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:53:40 2022] PUT /identity/v3/projects/82a9c61f73594105844658e5abf067dc/users/a721daa9088941ed9a4b85a42deeab0c/roles/11f8731756fe4caca13efa61b71b11c5 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:53:41.071481 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feddd0b4-e7da-41bc-8783-f869f04b36a5 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:41.072069 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feddd0b4-e7da-41bc-8783-f869f04b36a5 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:41.073194 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feddd0b4-e7da-41bc-8783-f869f04b36a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:41.109016 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-feddd0b4-e7da-41bc-8783-f869f04b36a5 None None] MFA Rules not processed for user `a721daa9088941ed9a4b85a42deeab0c`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:53:41.110648 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-feddd0b4-e7da-41bc-8783-f869f04b36a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:41.196006 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1303/2630] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:53:41 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:53:41.201208 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341ac9f4-0908-4f4f-a71e-45f3fd6ba8e9 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:41.270278 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341ac9f4-0908-4f4f-a71e-45f3fd6ba8e9 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:41.275483 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f659e3-381d-45ce-8381-df8985912d47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:41.276918 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f659e3-381d-45ce-8381-df8985912d47 admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:41.277243 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f659e3-381d-45ce-8381-df8985912d47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:41.277527 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f659e3-381d-45ce-8381-df8985912d47 admin admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:41.278195 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f659e3-381d-45ce-8381-df8985912d47 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:41.280138 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f659e3-381d-45ce-8381-df8985912d47 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:41.312122 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1328/2631] 10.222.0.89 () {60 vars in 1153 bytes} [Thu Jul 28 17:53:41 2022] POST /identity/v3/projects => generated 377 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:53:41.316812 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feddd0b4-e7da-41bc-8783-f869f04b36a5 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:41.328669 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feddd0b4-e7da-41bc-8783-f869f04b36a5 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:41.377312 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:41.397854 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6f0f5a-75ab-4581-9eba-94828ebc42fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:41.398484 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6f0f5a-75ab-4581-9eba-94828ebc42fc admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:41.398767 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6f0f5a-75ab-4581-9eba-94828ebc42fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:41.399044 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6f0f5a-75ab-4581-9eba-94828ebc42fc admin admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:41.399726 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6f0f5a-75ab-4581-9eba-94828ebc42fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:41.402655 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6f0f5a-75ab-4581-9eba-94828ebc42fc admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:41.427568 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1304/2632] 10.222.0.89 () {60 vars in 1144 bytes} [Thu Jul 28 17:53:41 2022] POST /identity/v3/users => generated 295 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 17:53:41.433718 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f659e3-381d-45ce-8381-df8985912d47 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:41.444801 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f659e3-381d-45ce-8381-df8985912d47 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:41.450747 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3b803d-cdea-4e32-bf84-89c764eec7d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:41.451537 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3b803d-cdea-4e32-bf84-89c764eec7d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:41.451905 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3b803d-cdea-4e32-bf84-89c764eec7d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:41.452282 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3b803d-cdea-4e32-bf84-89c764eec7d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:41.453258 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3b803d-cdea-4e32-bf84-89c764eec7d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:41.455047 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3b803d-cdea-4e32-bf84-89c764eec7d3 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:41.468310 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1329/2633] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:53:41 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:53:41.472641 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6f0f5a-75ab-4581-9eba-94828ebc42fc admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:41.506009 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6f0f5a-75ab-4581-9eba-94828ebc42fc admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:41.513392 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a241507-d111-40cc-b97b-9f88d5620f85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:41.514223 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a241507-d111-40cc-b97b-9f88d5620f85 admin admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:41.514509 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a241507-d111-40cc-b97b-9f88d5620f85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:41.514783 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a241507-d111-40cc-b97b-9f88d5620f85 admin admin] PATH_INFO: `/v3/projects/59151d06ba1f4c7b8a8dd0b34af9729b/users/9989322a7fcc4a5a96eef5ffd403495f/roles/73e457231a824ea98c28d22610be640c` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:41.550824 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a241507-d111-40cc-b97b-9f88d5620f85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=59151d06ba1f4c7b8a8dd0b34af9729b, user_id=9989322a7fcc4a5a96eef5ffd403495f, role_id=73e457231a824ea98c28d22610be640c)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:41.554831 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a241507-d111-40cc-b97b-9f88d5620f85 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:41.578135 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1305/2634] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:53:41 2022] PUT /identity/v3/projects/59151d06ba1f4c7b8a8dd0b34af9729b/users/9989322a7fcc4a5a96eef5ffd403495f/roles/73e457231a824ea98c28d22610be640c => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 17:53:41.582940 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3b803d-cdea-4e32-bf84-89c764eec7d3 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:41.599630 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3b803d-cdea-4e32-bf84-89c764eec7d3 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:41.638789 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:41.685444 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959146f5-588b-494d-a2b3-85fb8d008c9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:41.686522 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959146f5-588b-494d-a2b3-85fb8d008c9a admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:41.687023 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959146f5-588b-494d-a2b3-85fb8d008c9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:41.687499 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959146f5-588b-494d-a2b3-85fb8d008c9a admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:41.688239 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959146f5-588b-494d-a2b3-85fb8d008c9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:41.690054 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959146f5-588b-494d-a2b3-85fb8d008c9a admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:41.704738 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1330/2635] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:53:41 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 17:53:41.711238 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a241507-d111-40cc-b97b-9f88d5620f85 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:41.733729 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a241507-d111-40cc-b97b-9f88d5620f85 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:41.746827 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6328aabe-0db0-43f4-b1c7-1a3dba51da8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:41.748008 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6328aabe-0db0-43f4-b1c7-1a3dba51da8b admin admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:41.748476 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6328aabe-0db0-43f4-b1c7-1a3dba51da8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:41.749330 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6328aabe-0db0-43f4-b1c7-1a3dba51da8b admin admin] PATH_INFO: `/v3/projects/59151d06ba1f4c7b8a8dd0b34af9729b/users/9989322a7fcc4a5a96eef5ffd403495f/roles/73e457231a824ea98c28d22610be640c` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:41.752998 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6328aabe-0db0-43f4-b1c7-1a3dba51da8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=59151d06ba1f4c7b8a8dd0b34af9729b, user_id=9989322a7fcc4a5a96eef5ffd403495f, role_id=73e457231a824ea98c28d22610be640c)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:41.756120 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6328aabe-0db0-43f4-b1c7-1a3dba51da8b admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:41.771994 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1306/2636] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:53:41 2022] PUT /identity/v3/projects/59151d06ba1f4c7b8a8dd0b34af9729b/users/9989322a7fcc4a5a96eef5ffd403495f/roles/73e457231a824ea98c28d22610be640c => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:53:41.786631 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2097a778-fbd7-4da5-9b3c-df4350ba492a None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:41.787077 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2097a778-fbd7-4da5-9b3c-df4350ba492a None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:41.787599 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2097a778-fbd7-4da5-9b3c-df4350ba492a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:41.847305 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-2097a778-fbd7-4da5-9b3c-df4350ba492a None None] MFA Rules not processed for user `9989322a7fcc4a5a96eef5ffd403495f`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:53:41.848810 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-2097a778-fbd7-4da5-9b3c-df4350ba492a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:41.984741 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1331/2637] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:53:41 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:53:42.005584 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6328aabe-0db0-43f4-b1c7-1a3dba51da8b admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:42.007429 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-6328aabe-0db0-43f4-b1c7-1a3dba51da8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:42.092816 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6328aabe-0db0-43f4-b1c7-1a3dba51da8b admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:42.097182 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c64708-7e94-4e4d-b633-27212178e0b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96d512d0c84240918dae6c4a3e8b7b79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:42.098012 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c64708-7e94-4e4d-b633-27212178e0b5 service cinder] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:42.098459 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c64708-7e94-4e4d-b633-27212178e0b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:42.098912 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c64708-7e94-4e4d-b633-27212178e0b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:42.109343 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c64708-7e94-4e4d-b633-27212178e0b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:42.111000 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c64708-7e94-4e4d-b633-27212178e0b5 service cinder] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:42.122101 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1307/2638] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:53:42 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:53:42.660340 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2097a778-fbd7-4da5-9b3c-df4350ba492a None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:42.733039 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2097a778-fbd7-4da5-9b3c-df4350ba492a None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:42.738375 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60b59d5-6f77-48bf-8ec9-1b3d36562a93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:42.739026 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60b59d5-6f77-48bf-8ec9-1b3d36562a93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:42.739315 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60b59d5-6f77-48bf-8ec9-1b3d36562a93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:42.739957 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60b59d5-6f77-48bf-8ec9-1b3d36562a93 admin admin] PATH_INFO: `/v3/users/a721daa9088941ed9a4b85a42deeab0c` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:42.741244 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60b59d5-6f77-48bf-8ec9-1b3d36562a93 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a721daa9088941ed9a4b85a42deeab0c)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:42.743210 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60b59d5-6f77-48bf-8ec9-1b3d36562a93 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:42.797461 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-a60b59d5-6f77-48bf-8ec9-1b3d36562a93 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a721daa9088941ed9a4b85a42deeab0c'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:53:42.803029 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:42.814671 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-a60b59d5-6f77-48bf-8ec9-1b3d36562a93 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a721daa9088941ed9a4b85a42deeab0c'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:53:42.814671 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-a60b59d5-6f77-48bf-8ec9-1b3d36562a93 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a721daa9088941ed9a4b85a42deeab0c'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:53:42.829785 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-a60b59d5-6f77-48bf-8ec9-1b3d36562a93 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a721daa9088941ed9a4b85a42deeab0c'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:53:42.844183 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-a60b59d5-6f77-48bf-8ec9-1b3d36562a93 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a721daa9088941ed9a4b85a42deeab0c'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:53:42.855084 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1332/2639] 10.222.0.89 () {58 vars in 1224 bytes} [Thu Jul 28 17:53:42 2022] DELETE /identity/v3/users/a721daa9088941ed9a4b85a42deeab0c => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:53:42.856782 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80c1e3f-fd7c-49f5-8f82-a7f117e4b5c6 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:42.857349 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80c1e3f-fd7c-49f5-8f82-a7f117e4b5c6 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:42.857555 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80c1e3f-fd7c-49f5-8f82-a7f117e4b5c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:42.878374 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-f80c1e3f-fd7c-49f5-8f82-a7f117e4b5c6 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:53:42.879797 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-f80c1e3f-fd7c-49f5-8f82-a7f117e4b5c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:42.957357 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1308/2640] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:53:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:53:42.966454 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60b59d5-6f77-48bf-8ec9-1b3d36562a93 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:42.968573 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-a60b59d5-6f77-48bf-8ec9-1b3d36562a93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:43.043002 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60b59d5-6f77-48bf-8ec9-1b3d36562a93 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:43.048080 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7d14a8-5e78-4af2-bc90-82f8a52b2501 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76817cd541294331b0023069092e8e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:43.049306 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7d14a8-5e78-4af2-bc90-82f8a52b2501 service neutron] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:43.049306 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7d14a8-5e78-4af2-bc90-82f8a52b2501 service neutron] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:43.049306 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7d14a8-5e78-4af2-bc90-82f8a52b2501 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:43.060175 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7d14a8-5e78-4af2-bc90-82f8a52b2501 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:43.061708 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7d14a8-5e78-4af2-bc90-82f8a52b2501 service neutron] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:43.073190 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1333/2641] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:53:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 17:53:43.397291 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80c1e3f-fd7c-49f5-8f82-a7f117e4b5c6 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:43.398741 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-f80c1e3f-fd7c-49f5-8f82-a7f117e4b5c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:43.411788 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80c1e3f-fd7c-49f5-8f82-a7f117e4b5c6 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:43.425902 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f3d49b-c2cf-4b34-9e93-50864b55297d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:43.426746 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f3d49b-c2cf-4b34-9e93-50864b55297d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:43.427389 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f3d49b-c2cf-4b34-9e93-50864b55297d admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:43.427765 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f3d49b-c2cf-4b34-9e93-50864b55297d admin admin] PATH_INFO: `/v3/projects/82a9c61f73594105844658e5abf067dc` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:43.429494 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f3d49b-c2cf-4b34-9e93-50864b55297d admin admin] RBAC: Authorizing `identity:delete_project(project_id=82a9c61f73594105844658e5abf067dc)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:43.431742 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f3d49b-c2cf-4b34-9e93-50864b55297d admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:43.478514 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:43.492294 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-56f3d49b-c2cf-4b34-9e93-50864b55297d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82a9c61f73594105844658e5abf067dc', 'request_id': 'req-56f3d49b-c2cf-4b34-9e93-50864b55297d'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:53:43.500815 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-56f3d49b-c2cf-4b34-9e93-50864b55297d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82a9c61f73594105844658e5abf067dc', 'request_id': 'req-56f3d49b-c2cf-4b34-9e93-50864b55297d'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:53:43.508488 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-56f3d49b-c2cf-4b34-9e93-50864b55297d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82a9c61f73594105844658e5abf067dc', 'request_id': 'req-56f3d49b-c2cf-4b34-9e93-50864b55297d'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:53:43.530014 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-56f3d49b-c2cf-4b34-9e93-50864b55297d admin admin] The token cache is being invalidate because project 82a9c61f73594105844658e5abf067dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80325) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 17:53:43.530327 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-56f3d49b-c2cf-4b34-9e93-50864b55297d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:53:43.531168 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-56f3d49b-c2cf-4b34-9e93-50864b55297d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:53:43.533098 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1309/2642] 10.222.0.89 () {58 vars in 1233 bytes} [Thu Jul 28 17:53:43 2022] DELETE /identity/v3/projects/82a9c61f73594105844658e5abf067dc => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:53:43.537072 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7d14a8-5e78-4af2-bc90-82f8a52b2501 service neutron] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:43.547969 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7d14a8-5e78-4af2-bc90-82f8a52b2501 service neutron] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:43.633884 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee6a7e9-327d-4451-b386-8a725006d36c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:43.635744 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee6a7e9-327d-4451-b386-8a725006d36c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:43.636680 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee6a7e9-327d-4451-b386-8a725006d36c admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:43.637196 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee6a7e9-327d-4451-b386-8a725006d36c admin admin] PATH_INFO: `/v3/users/9989322a7fcc4a5a96eef5ffd403495f` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:43.638867 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee6a7e9-327d-4451-b386-8a725006d36c admin admin] RBAC: Authorizing `identity:delete_user(user_id=9989322a7fcc4a5a96eef5ffd403495f)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:43.641452 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee6a7e9-327d-4451-b386-8a725006d36c admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:43.702986 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-5ee6a7e9-327d-4451-b386-8a725006d36c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9989322a7fcc4a5a96eef5ffd403495f'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:53:43.724053 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-5ee6a7e9-327d-4451-b386-8a725006d36c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9989322a7fcc4a5a96eef5ffd403495f'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:53:43.724867 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-5ee6a7e9-327d-4451-b386-8a725006d36c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9989322a7fcc4a5a96eef5ffd403495f'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:53:43.735015 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-5ee6a7e9-327d-4451-b386-8a725006d36c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9989322a7fcc4a5a96eef5ffd403495f'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:53:43.746719 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:DELETE=12 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:43.753475 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-5ee6a7e9-327d-4451-b386-8a725006d36c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9989322a7fcc4a5a96eef5ffd403495f'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:53:43.764502 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1334/2643] 10.222.0.89 () {58 vars in 1224 bytes} [Thu Jul 28 17:53:43 2022] DELETE /identity/v3/users/9989322a7fcc4a5a96eef5ffd403495f => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:53:44.109337 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f3d49b-c2cf-4b34-9e93-50864b55297d admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:44.110947 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-56f3d49b-c2cf-4b34-9e93-50864b55297d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:44.141116 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f3d49b-c2cf-4b34-9e93-50864b55297d admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:44.256622 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87d9234-f3c9-46b7-8455-0f4beb9db82d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:44.257866 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87d9234-f3c9-46b7-8455-0f4beb9db82d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:44.258252 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87d9234-f3c9-46b7-8455-0f4beb9db82d admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:44.258785 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87d9234-f3c9-46b7-8455-0f4beb9db82d admin admin] PATH_INFO: `/v3/projects/59151d06ba1f4c7b8a8dd0b34af9729b` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:44.260399 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87d9234-f3c9-46b7-8455-0f4beb9db82d admin admin] RBAC: Authorizing `identity:delete_project(project_id=59151d06ba1f4c7b8a8dd0b34af9729b)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:44.263008 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87d9234-f3c9-46b7-8455-0f4beb9db82d admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:44.339405 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-b87d9234-f3c9-46b7-8455-0f4beb9db82d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59151d06ba1f4c7b8a8dd0b34af9729b', 'request_id': 'req-b87d9234-f3c9-46b7-8455-0f4beb9db82d'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:53:44.358181 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-b87d9234-f3c9-46b7-8455-0f4beb9db82d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59151d06ba1f4c7b8a8dd0b34af9729b', 'request_id': 'req-b87d9234-f3c9-46b7-8455-0f4beb9db82d'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:53:44.365538 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-b87d9234-f3c9-46b7-8455-0f4beb9db82d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59151d06ba1f4c7b8a8dd0b34af9729b', 'request_id': 'req-b87d9234-f3c9-46b7-8455-0f4beb9db82d'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:53:44.381775 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-b87d9234-f3c9-46b7-8455-0f4beb9db82d admin admin] The token cache is being invalidate because project 59151d06ba1f4c7b8a8dd0b34af9729b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80325) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 17:53:44.382083 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-b87d9234-f3c9-46b7-8455-0f4beb9db82d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:53:44.382731 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-b87d9234-f3c9-46b7-8455-0f4beb9db82d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:53:44.384547 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1310/2644] 10.222.0.89 () {58 vars in 1233 bytes} [Thu Jul 28 17:53:44 2022] DELETE /identity/v3/projects/59151d06ba1f4c7b8a8dd0b34af9729b => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:53:44.389744 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effa1e58-1406-4299-a9a3-2a4ed6eaea71 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:44.390137 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effa1e58-1406-4299-a9a3-2a4ed6eaea71 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:44.390456 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effa1e58-1406-4299-a9a3-2a4ed6eaea71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:44.411781 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-effa1e58-1406-4299-a9a3-2a4ed6eaea71 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:53:44.413240 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-effa1e58-1406-4299-a9a3-2a4ed6eaea71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:44.483469 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1335/2645] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:53:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:53:44.558973 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1e00d5-5be0-4d49-9fed-0b6b126d57a1 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:44.559269 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1e00d5-5be0-4d49-9fed-0b6b126d57a1 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:44.559501 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1e00d5-5be0-4d49-9fed-0b6b126d57a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:44.583728 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-1e1e00d5-5be0-4d49-9fed-0b6b126d57a1 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:53:44.584977 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-1e1e00d5-5be0-4d49-9fed-0b6b126d57a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:44.594173 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1311/2646] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:53:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:53:44.598463 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effa1e58-1406-4299-a9a3-2a4ed6eaea71 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:44.609321 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effa1e58-1406-4299-a9a3-2a4ed6eaea71 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:44.613999 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fed3a4e-c5dd-4460-b0cf-e3096b64842d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:44.614632 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fed3a4e-c5dd-4460-b0cf-e3096b64842d admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:44.614943 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fed3a4e-c5dd-4460-b0cf-e3096b64842d admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:44.615224 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fed3a4e-c5dd-4460-b0cf-e3096b64842d admin admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:44.615930 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fed3a4e-c5dd-4460-b0cf-e3096b64842d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:44.616283 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fed3a4e-c5dd-4460-b0cf-e3096b64842d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:44.618006 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fed3a4e-c5dd-4460-b0cf-e3096b64842d admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:44.634133 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1336/2647] 10.222.0.89 () {58 vars in 1153 bytes} [Thu Jul 28 17:53:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:44.639853 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1e00d5-5be0-4d49-9fed-0b6b126d57a1 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:44.650081 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1e00d5-5be0-4d49-9fed-0b6b126d57a1 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:44.655701 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ec49b4-8695-4104-8924-1d2536f24aaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:44.656341 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ec49b4-8695-4104-8924-1d2536f24aaf admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:44.656625 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ec49b4-8695-4104-8924-1d2536f24aaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:44.656891 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ec49b4-8695-4104-8924-1d2536f24aaf admin admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:44.657598 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ec49b4-8695-4104-8924-1d2536f24aaf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:44.659553 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ec49b4-8695-4104-8924-1d2536f24aaf admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:44.703684 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1312/2648] 10.222.0.89 () {60 vars in 1153 bytes} [Thu Jul 28 17:53:44 2022] POST /identity/v3/projects => generated 385 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:53:44.707787 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fed3a4e-c5dd-4460-b0cf-e3096b64842d admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:44.719359 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fed3a4e-c5dd-4460-b0cf-e3096b64842d admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:44.821355 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039dca03-6975-43ee-8b96-2c5d545d16f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:44.823891 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039dca03-6975-43ee-8b96-2c5d545d16f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:44.824275 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039dca03-6975-43ee-8b96-2c5d545d16f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:44.824794 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039dca03-6975-43ee-8b96-2c5d545d16f2 admin admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:44.826052 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039dca03-6975-43ee-8b96-2c5d545d16f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:44.830286 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039dca03-6975-43ee-8b96-2c5d545d16f2 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:44.868652 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1337/2649] 10.222.0.89 () {60 vars in 1144 bytes} [Thu Jul 28 17:53:44 2022] POST /identity/v3/users => generated 293 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 17:53:44.873649 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ec49b4-8695-4104-8924-1d2536f24aaf admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:44.890831 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ec49b4-8695-4104-8924-1d2536f24aaf admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:44.905396 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cedd6c7-28d5-4969-aa78-75c39b114217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:44.906593 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cedd6c7-28d5-4969-aa78-75c39b114217 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:44.906593 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cedd6c7-28d5-4969-aa78-75c39b114217 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:44.906861 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cedd6c7-28d5-4969-aa78-75c39b114217 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:44.907862 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cedd6c7-28d5-4969-aa78-75c39b114217 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:44.910211 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cedd6c7-28d5-4969-aa78-75c39b114217 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:44.955195 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1313/2650] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:53:44 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:53:44.956700 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039dca03-6975-43ee-8b96-2c5d545d16f2 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:44.974856 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039dca03-6975-43ee-8b96-2c5d545d16f2 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:44.982233 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debf29bf-7ea5-4684-903b-50f715d9d5dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:44.982425 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debf29bf-7ea5-4684-903b-50f715d9d5dc admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:44.982733 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debf29bf-7ea5-4684-903b-50f715d9d5dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:44.983011 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debf29bf-7ea5-4684-903b-50f715d9d5dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:44.984907 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debf29bf-7ea5-4684-903b-50f715d9d5dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:44.987739 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debf29bf-7ea5-4684-903b-50f715d9d5dc admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:45.003776 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1338/2651] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:53:44 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 17:53:45.007891 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cedd6c7-28d5-4969-aa78-75c39b114217 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:45.019454 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cedd6c7-28d5-4969-aa78-75c39b114217 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:45.024554 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67773bc6-b327-4b75-a930-b4d2d4147655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:45.025377 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67773bc6-b327-4b75-a930-b4d2d4147655 admin admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:45.025813 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67773bc6-b327-4b75-a930-b4d2d4147655 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:45.026140 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67773bc6-b327-4b75-a930-b4d2d4147655 admin admin] PATH_INFO: `/v3/projects/1f25be3d0b364e88bb1602b80d264a61/users/c18d0ac8956b4b62b7a84551f6e969ec/roles/11f8731756fe4caca13efa61b71b11c5` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:45.038741 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:45.043918 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67773bc6-b327-4b75-a930-b4d2d4147655 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f25be3d0b364e88bb1602b80d264a61, user_id=c18d0ac8956b4b62b7a84551f6e969ec, role_id=11f8731756fe4caca13efa61b71b11c5)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:45.046896 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67773bc6-b327-4b75-a930-b4d2d4147655 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:45.079497 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1314/2652] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:53:45 2022] PUT /identity/v3/projects/1f25be3d0b364e88bb1602b80d264a61/users/c18d0ac8956b4b62b7a84551f6e969ec/roles/11f8731756fe4caca13efa61b71b11c5 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:53:45.082248 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ec73c1-c9de-46bc-bec8-8ec28b19fa49 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:45.083299 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ec73c1-c9de-46bc-bec8-8ec28b19fa49 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:45.083828 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ec73c1-c9de-46bc-bec8-8ec28b19fa49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:45.145937 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-79ec73c1-c9de-46bc-bec8-8ec28b19fa49 None None] MFA Rules not processed for user `c18d0ac8956b4b62b7a84551f6e969ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:53:45.147409 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-79ec73c1-c9de-46bc-bec8-8ec28b19fa49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:45.203426 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:45.236791 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1339/2653] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:53:45 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:53:45.241961 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67773bc6-b327-4b75-a930-b4d2d4147655 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:45.315650 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67773bc6-b327-4b75-a930-b4d2d4147655 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:45.321380 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2ced6f-518c-444d-827f-fd6aed33cf3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:45.324096 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2ced6f-518c-444d-827f-fd6aed33cf3d admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:45.326269 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2ced6f-518c-444d-827f-fd6aed33cf3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:45.326549 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2ced6f-518c-444d-827f-fd6aed33cf3d admin admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:45.327232 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2ced6f-518c-444d-827f-fd6aed33cf3d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:45.329129 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2ced6f-518c-444d-827f-fd6aed33cf3d admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:45.371904 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1315/2654] 10.222.0.89 () {60 vars in 1153 bytes} [Thu Jul 28 17:53:45 2022] POST /identity/v3/projects => generated 383 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:53:45.373071 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ec73c1-c9de-46bc-bec8-8ec28b19fa49 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:45.385665 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ec73c1-c9de-46bc-bec8-8ec28b19fa49 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:45.477497 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc84f47-37cd-4cb9-922b-ed961c68a50d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:45.478457 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc84f47-37cd-4cb9-922b-ed961c68a50d admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:45.478805 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc84f47-37cd-4cb9-922b-ed961c68a50d admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:45.479090 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc84f47-37cd-4cb9-922b-ed961c68a50d admin admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:45.480194 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc84f47-37cd-4cb9-922b-ed961c68a50d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:45.482295 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc84f47-37cd-4cb9-922b-ed961c68a50d admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:45.514431 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1340/2655] 10.222.0.89 () {60 vars in 1144 bytes} [Thu Jul 28 17:53:45 2022] POST /identity/v3/users => generated 298 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:53:45.515952 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2ced6f-518c-444d-827f-fd6aed33cf3d admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:45.529972 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2ced6f-518c-444d-827f-fd6aed33cf3d admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:45.535376 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ba0b56-059e-4749-8177-f005d8cd1644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:45.536156 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ba0b56-059e-4749-8177-f005d8cd1644 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:45.536542 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ba0b56-059e-4749-8177-f005d8cd1644 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:45.536915 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ba0b56-059e-4749-8177-f005d8cd1644 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:45.537768 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ba0b56-059e-4749-8177-f005d8cd1644 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:45.539388 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ba0b56-059e-4749-8177-f005d8cd1644 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:45.552925 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1316/2656] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:53:45 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:53:45.559236 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc84f47-37cd-4cb9-922b-ed961c68a50d admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:45.573508 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc84f47-37cd-4cb9-922b-ed961c68a50d admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:45.582897 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207001a0-7a24-4977-986a-0c9496378a93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:45.583593 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207001a0-7a24-4977-986a-0c9496378a93 admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:45.583875 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207001a0-7a24-4977-986a-0c9496378a93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:45.584148 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207001a0-7a24-4977-986a-0c9496378a93 admin admin] PATH_INFO: `/v3/projects/80f2d1076292419ca77ba215411b3798/users/97bb339d11be4d1c87c860c753379246/roles/73e457231a824ea98c28d22610be640c` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:45.603257 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207001a0-7a24-4977-986a-0c9496378a93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80f2d1076292419ca77ba215411b3798, user_id=97bb339d11be4d1c87c860c753379246, role_id=73e457231a824ea98c28d22610be640c)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:45.606792 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207001a0-7a24-4977-986a-0c9496378a93 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:45.624690 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1341/2657] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:53:45 2022] PUT /identity/v3/projects/80f2d1076292419ca77ba215411b3798/users/97bb339d11be4d1c87c860c753379246/roles/73e457231a824ea98c28d22610be640c => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:53:45.629535 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ba0b56-059e-4749-8177-f005d8cd1644 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:45.641295 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ba0b56-059e-4749-8177-f005d8cd1644 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:45.713376 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975857f1-5135-47c4-b26a-6e0b7f17c7d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:45.714707 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975857f1-5135-47c4-b26a-6e0b7f17c7d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:45.715150 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975857f1-5135-47c4-b26a-6e0b7f17c7d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:45.715582 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975857f1-5135-47c4-b26a-6e0b7f17c7d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:45.719084 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975857f1-5135-47c4-b26a-6e0b7f17c7d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:45.721195 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975857f1-5135-47c4-b26a-6e0b7f17c7d4 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:45.734639 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1317/2658] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:53:45 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 17:53:45.738358 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207001a0-7a24-4977-986a-0c9496378a93 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:45.751800 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207001a0-7a24-4977-986a-0c9496378a93 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:45.757619 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0910cc0-abf7-42b9-9543-e32b1336b500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:45.758291 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0910cc0-abf7-42b9-9543-e32b1336b500 admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:45.758572 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0910cc0-abf7-42b9-9543-e32b1336b500 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:45.758847 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0910cc0-abf7-42b9-9543-e32b1336b500 admin admin] PATH_INFO: `/v3/projects/80f2d1076292419ca77ba215411b3798/users/97bb339d11be4d1c87c860c753379246/roles/73e457231a824ea98c28d22610be640c` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:45.761018 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0910cc0-abf7-42b9-9543-e32b1336b500 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80f2d1076292419ca77ba215411b3798, user_id=97bb339d11be4d1c87c860c753379246, role_id=73e457231a824ea98c28d22610be640c)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:45.763798 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0910cc0-abf7-42b9-9543-e32b1336b500 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:45.779228 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1342/2659] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:53:45 2022] PUT /identity/v3/projects/80f2d1076292419ca77ba215411b3798/users/97bb339d11be4d1c87c860c753379246/roles/73e457231a824ea98c28d22610be640c => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:53:45.789341 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1529b8-9e1b-4037-8c2c-28198a1cdea7 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:45.789981 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1529b8-9e1b-4037-8c2c-28198a1cdea7 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:45.790572 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1529b8-9e1b-4037-8c2c-28198a1cdea7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:45.832419 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-1b1529b8-9e1b-4037-8c2c-28198a1cdea7 None None] MFA Rules not processed for user `97bb339d11be4d1c87c860c753379246`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:53:45.833921 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-1b1529b8-9e1b-4037-8c2c-28198a1cdea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:45.910594 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:45.948281 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1318/2660] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:53:45 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:53:45.959783 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0910cc0-abf7-42b9-9543-e32b1336b500 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:45.961672 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-b0910cc0-abf7-42b9-9543-e32b1336b500 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:46.036562 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0910cc0-abf7-42b9-9543-e32b1336b500 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:46.047763 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233976ef-9996-4dd5-bfeb-e15f465112ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96d512d0c84240918dae6c4a3e8b7b79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:46.048414 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233976ef-9996-4dd5-bfeb-e15f465112ff service cinder] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:46.048702 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233976ef-9996-4dd5-bfeb-e15f465112ff service cinder] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:46.048987 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233976ef-9996-4dd5-bfeb-e15f465112ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:46.117343 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:46.128711 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233976ef-9996-4dd5-bfeb-e15f465112ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:46.130263 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233976ef-9996-4dd5-bfeb-e15f465112ff service cinder] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:46.154011 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1343/2661] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:53:45 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:53:50.803982 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1529b8-9e1b-4037-8c2c-28198a1cdea7 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:50.805462 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-1b1529b8-9e1b-4037-8c2c-28198a1cdea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:50.824449 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1529b8-9e1b-4037-8c2c-28198a1cdea7 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:50.832574 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b424fd0-80ec-47bc-969e-8392ae2290d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96d512d0c84240918dae6c4a3e8b7b79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:50.833412 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b424fd0-80ec-47bc-969e-8392ae2290d8 service cinder] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:50.833770 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b424fd0-80ec-47bc-969e-8392ae2290d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:50.834156 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b424fd0-80ec-47bc-969e-8392ae2290d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:50.846105 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b424fd0-80ec-47bc-969e-8392ae2290d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:50.847770 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b424fd0-80ec-47bc-969e-8392ae2290d8 service cinder] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:50.858626 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1319/2662] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:53:50 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:53:51.402221 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233976ef-9996-4dd5-bfeb-e15f465112ff service cinder] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:51.403700 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-233976ef-9996-4dd5-bfeb-e15f465112ff service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:51.464264 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233976ef-9996-4dd5-bfeb-e15f465112ff service cinder] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:51.467862 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fa313d-36a2-4471-ad62-9a6d37b40fe8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76817cd541294331b0023069092e8e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:51.468479 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fa313d-36a2-4471-ad62-9a6d37b40fe8 service neutron] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:51.468761 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fa313d-36a2-4471-ad62-9a6d37b40fe8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:51.469034 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fa313d-36a2-4471-ad62-9a6d37b40fe8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:51.470844 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-58fa313d-36a2-4471-ad62-9a6d37b40fe8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:51.529830 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fa313d-36a2-4471-ad62-9a6d37b40fe8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:51.531294 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fa313d-36a2-4471-ad62-9a6d37b40fe8 service neutron] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:51.540285 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1344/2663] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:53:51 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:53:55.446174 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be140ff3-7e81-48f3-b6d9-be1e917f8d81 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:55.446669 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be140ff3-7e81-48f3-b6d9-be1e917f8d81 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:55.446784 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be140ff3-7e81-48f3-b6d9-be1e917f8d81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:55.467888 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-be140ff3-7e81-48f3-b6d9-be1e917f8d81 None None] MFA Rules not processed for user `c44387d13a694ee6bb47a13906003330`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:53:55.469113 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-be140ff3-7e81-48f3-b6d9-be1e917f8d81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:55.530787 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1320/2664] 10.222.0.89 () {58 vars in 967 bytes} [Thu Jul 28 17:53:55 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:53:55.543414 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fa313d-36a2-4471-ad62-9a6d37b40fe8 service neutron] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:55.544783 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-58fa313d-36a2-4471-ad62-9a6d37b40fe8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:55.616774 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fa313d-36a2-4471-ad62-9a6d37b40fe8 service neutron] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:55.622005 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfab4558-8969-4ab2-92bc-e4d291c36d6e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c59473b01bf4433ba3b111114fc048b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:55.622678 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfab4558-8969-4ab2-92bc-e4d291c36d6e service swift] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:55.622963 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfab4558-8969-4ab2-92bc-e4d291c36d6e service swift] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:55.623236 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfab4558-8969-4ab2-92bc-e4d291c36d6e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:55.631607 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfab4558-8969-4ab2-92bc-e4d291c36d6e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:55.633304 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfab4558-8969-4ab2-92bc-e4d291c36d6e service swift] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:55.642985 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1345/2665] 10.222.0.89 () {60 vars in 1384 bytes} [Thu Jul 28 17:53:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 17:53:56.087711 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be140ff3-7e81-48f3-b6d9-be1e917f8d81 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:56.156527 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be140ff3-7e81-48f3-b6d9-be1e917f8d81 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:56.163047 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2116eb4-ef3c-4e1a-9ffc-85255c9d1a29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:56.163730 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2116eb4-ef3c-4e1a-9ffc-85255c9d1a29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:56.164530 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2116eb4-ef3c-4e1a-9ffc-85255c9d1a29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:56.164826 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2116eb4-ef3c-4e1a-9ffc-85255c9d1a29 admin admin] PATH_INFO: `/v3/users/c18d0ac8956b4b62b7a84551f6e969ec` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:56.166594 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2116eb4-ef3c-4e1a-9ffc-85255c9d1a29 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c18d0ac8956b4b62b7a84551f6e969ec)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:56.169953 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2116eb4-ef3c-4e1a-9ffc-85255c9d1a29 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:56.230562 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-a2116eb4-ef3c-4e1a-9ffc-85255c9d1a29 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c18d0ac8956b4b62b7a84551f6e969ec'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:53:56.238167 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-a2116eb4-ef3c-4e1a-9ffc-85255c9d1a29 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c18d0ac8956b4b62b7a84551f6e969ec'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:53:56.247306 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-a2116eb4-ef3c-4e1a-9ffc-85255c9d1a29 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c18d0ac8956b4b62b7a84551f6e969ec'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:53:56.247939 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-a2116eb4-ef3c-4e1a-9ffc-85255c9d1a29 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c18d0ac8956b4b62b7a84551f6e969ec'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:53:56.248746 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-a2116eb4-ef3c-4e1a-9ffc-85255c9d1a29 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c18d0ac8956b4b62b7a84551f6e969ec'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:53:56.262257 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:56.270859 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1321/2666] 10.222.0.89 () {58 vars in 1224 bytes} [Thu Jul 28 17:53:56 2022] DELETE /identity/v3/users/c18d0ac8956b4b62b7a84551f6e969ec => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:53:56.275970 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0accb10-098e-4c7a-b175-429c24e40c2a None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:56.277238 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0accb10-098e-4c7a-b175-429c24e40c2a None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:56.277464 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0accb10-098e-4c7a-b175-429c24e40c2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:56.296551 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-a0accb10-098e-4c7a-b175-429c24e40c2a None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:53:56.297798 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-a0accb10-098e-4c7a-b175-429c24e40c2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:56.339806 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:56.373638 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1346/2667] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:53:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 17:53:56.384716 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2116eb4-ef3c-4e1a-9ffc-85255c9d1a29 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:56.387166 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-a2116eb4-ef3c-4e1a-9ffc-85255c9d1a29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:56.451208 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2116eb4-ef3c-4e1a-9ffc-85255c9d1a29 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:56.455024 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fe1b45-2b13-4ce2-a3db-6f04d62fb0fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76817cd541294331b0023069092e8e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:56.455640 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fe1b45-2b13-4ce2-a3db-6f04d62fb0fc service neutron] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:56.455919 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fe1b45-2b13-4ce2-a3db-6f04d62fb0fc service neutron] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:56.456188 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fe1b45-2b13-4ce2-a3db-6f04d62fb0fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:56.465359 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fe1b45-2b13-4ce2-a3db-6f04d62fb0fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:56.466774 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fe1b45-2b13-4ce2-a3db-6f04d62fb0fc service neutron] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:56.476206 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1322/2668] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:53:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:53:56.770071 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0accb10-098e-4c7a-b175-429c24e40c2a None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:56.771664 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-a0accb10-098e-4c7a-b175-429c24e40c2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:56.788232 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0accb10-098e-4c7a-b175-429c24e40c2a None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:56.795742 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111918dc-7dee-4708-a45d-3211a4b68341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:56.797129 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111918dc-7dee-4708-a45d-3211a4b68341 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:56.797974 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111918dc-7dee-4708-a45d-3211a4b68341 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:56.798336 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111918dc-7dee-4708-a45d-3211a4b68341 admin admin] PATH_INFO: `/v3/projects/1f25be3d0b364e88bb1602b80d264a61` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:56.800091 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111918dc-7dee-4708-a45d-3211a4b68341 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f25be3d0b364e88bb1602b80d264a61)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:56.802176 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111918dc-7dee-4708-a45d-3211a4b68341 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:56.859153 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-111918dc-7dee-4708-a45d-3211a4b68341 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f25be3d0b364e88bb1602b80d264a61', 'request_id': 'req-111918dc-7dee-4708-a45d-3211a4b68341'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:53:56.865502 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-111918dc-7dee-4708-a45d-3211a4b68341 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f25be3d0b364e88bb1602b80d264a61', 'request_id': 'req-111918dc-7dee-4708-a45d-3211a4b68341'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:53:56.873448 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-111918dc-7dee-4708-a45d-3211a4b68341 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f25be3d0b364e88bb1602b80d264a61', 'request_id': 'req-111918dc-7dee-4708-a45d-3211a4b68341'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:53:56.887467 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-111918dc-7dee-4708-a45d-3211a4b68341 admin admin] The token cache is being invalidate because project 1f25be3d0b364e88bb1602b80d264a61 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80324) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 17:53:56.887754 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-111918dc-7dee-4708-a45d-3211a4b68341 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:53:56.888467 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-111918dc-7dee-4708-a45d-3211a4b68341 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:53:56.890471 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1347/2669] 10.222.0.89 () {58 vars in 1233 bytes} [Thu Jul 28 17:53:56 2022] DELETE /identity/v3/projects/1f25be3d0b364e88bb1602b80d264a61 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:53:56.894370 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fe1b45-2b13-4ce2-a3db-6f04d62fb0fc service neutron] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:56.906188 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fe1b45-2b13-4ce2-a3db-6f04d62fb0fc service neutron] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:56.966417 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7464ea-2636-437e-986e-5c198838b350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:56.967066 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7464ea-2636-437e-986e-5c198838b350 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:56.967342 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7464ea-2636-437e-986e-5c198838b350 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:56.967617 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7464ea-2636-437e-986e-5c198838b350 admin admin] PATH_INFO: `/v3/users/97bb339d11be4d1c87c860c753379246` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:56.968813 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7464ea-2636-437e-986e-5c198838b350 admin admin] RBAC: Authorizing `identity:delete_user(user_id=97bb339d11be4d1c87c860c753379246)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:56.970789 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7464ea-2636-437e-986e-5c198838b350 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:57.016438 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-5b7464ea-2636-437e-986e-5c198838b350 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97bb339d11be4d1c87c860c753379246'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:53:57.023239 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-5b7464ea-2636-437e-986e-5c198838b350 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97bb339d11be4d1c87c860c753379246'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:53:57.030804 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-5b7464ea-2636-437e-986e-5c198838b350 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97bb339d11be4d1c87c860c753379246'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:53:57.031476 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-5b7464ea-2636-437e-986e-5c198838b350 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97bb339d11be4d1c87c860c753379246'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:53:57.032089 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-5b7464ea-2636-437e-986e-5c198838b350 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97bb339d11be4d1c87c860c753379246'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:53:57.050177 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1323/2670] 10.222.0.89 () {58 vars in 1224 bytes} [Thu Jul 28 17:53:56 2022] DELETE /identity/v3/users/97bb339d11be4d1c87c860c753379246 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 17:53:57.358105 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111918dc-7dee-4708-a45d-3211a4b68341 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:57.359553 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-111918dc-7dee-4708-a45d-3211a4b68341 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:57.373649 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111918dc-7dee-4708-a45d-3211a4b68341 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:57.447773 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf181ee7-1b5f-4991-a5e3-adae171caf1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:57.449876 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf181ee7-1b5f-4991-a5e3-adae171caf1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:57.450327 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf181ee7-1b5f-4991-a5e3-adae171caf1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:57.450604 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf181ee7-1b5f-4991-a5e3-adae171caf1a admin admin] PATH_INFO: `/v3/projects/80f2d1076292419ca77ba215411b3798` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:57.451871 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf181ee7-1b5f-4991-a5e3-adae171caf1a admin admin] RBAC: Authorizing `identity:delete_project(project_id=80f2d1076292419ca77ba215411b3798)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:57.453820 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf181ee7-1b5f-4991-a5e3-adae171caf1a admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:57.507217 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-bf181ee7-1b5f-4991-a5e3-adae171caf1a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80f2d1076292419ca77ba215411b3798', 'request_id': 'req-bf181ee7-1b5f-4991-a5e3-adae171caf1a'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:53:57.514312 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-bf181ee7-1b5f-4991-a5e3-adae171caf1a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80f2d1076292419ca77ba215411b3798', 'request_id': 'req-bf181ee7-1b5f-4991-a5e3-adae171caf1a'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:53:57.521230 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-bf181ee7-1b5f-4991-a5e3-adae171caf1a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80f2d1076292419ca77ba215411b3798', 'request_id': 'req-bf181ee7-1b5f-4991-a5e3-adae171caf1a'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:53:57.522901 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:57.537176 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-bf181ee7-1b5f-4991-a5e3-adae171caf1a admin admin] The token cache is being invalidate because project 80f2d1076292419ca77ba215411b3798 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80324) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 17:53:57.537487 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-bf181ee7-1b5f-4991-a5e3-adae171caf1a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:53:57.538177 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-bf181ee7-1b5f-4991-a5e3-adae171caf1a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:53:57.539897 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1348/2671] 10.222.0.89 () {58 vars in 1233 bytes} [Thu Jul 28 17:53:57 2022] DELETE /identity/v3/projects/80f2d1076292419ca77ba215411b3798 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:53:57.546465 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71b4a13-e2d9-439a-9c7b-be5027848555 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:57.546989 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71b4a13-e2d9-439a-9c7b-be5027848555 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:57.547439 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71b4a13-e2d9-439a-9c7b-be5027848555 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:57.570617 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-b71b4a13-e2d9-439a-9c7b-be5027848555 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:53:57.572075 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-b71b4a13-e2d9-439a-9c7b-be5027848555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:57.585180 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:57.639406 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1324/2672] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:53:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:53:57.647986 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110942a3-d43e-4239-a688-601ca9dd2f2d None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:57.649054 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110942a3-d43e-4239-a688-601ca9dd2f2d None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:57.649903 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110942a3-d43e-4239-a688-601ca9dd2f2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:57.663889 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71b4a13-e2d9-439a-9c7b-be5027848555 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:57.665713 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-b71b4a13-e2d9-439a-9c7b-be5027848555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:57.679903 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-110942a3-d43e-4239-a688-601ca9dd2f2d None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:53:57.682309 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-110942a3-d43e-4239-a688-601ca9dd2f2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:57.692009 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71b4a13-e2d9-439a-9c7b-be5027848555 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:57.692541 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1349/2673] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:53:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:53:57.703470 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110942a3-d43e-4239-a688-601ca9dd2f2d None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:57.706303 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9b1463-6e8a-470f-a4b7-448dc93e504b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:57.707150 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9b1463-6e8a-470f-a4b7-448dc93e504b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:57.707590 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9b1463-6e8a-470f-a4b7-448dc93e504b admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:57.708021 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9b1463-6e8a-470f-a4b7-448dc93e504b admin admin] PATH_INFO: `/v3/users/aedeb9e3e967402aa1458ceedf023dac` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:57.709516 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9b1463-6e8a-470f-a4b7-448dc93e504b admin admin] RBAC: Authorizing `identity:delete_user(user_id=aedeb9e3e967402aa1458ceedf023dac)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:57.711702 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9b1463-6e8a-470f-a4b7-448dc93e504b admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:57.712487 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110942a3-d43e-4239-a688-601ca9dd2f2d None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:57.718794 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc33006f-2b1e-435a-87e0-eaf493c40420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:57.719426 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc33006f-2b1e-435a-87e0-eaf493c40420 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:57.719717 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc33006f-2b1e-435a-87e0-eaf493c40420 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:57.719999 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc33006f-2b1e-435a-87e0-eaf493c40420 admin admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:57.720707 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc33006f-2b1e-435a-87e0-eaf493c40420 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:57.721041 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc33006f-2b1e-435a-87e0-eaf493c40420 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:57.722792 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc33006f-2b1e-435a-87e0-eaf493c40420 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:57.738883 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1350/2674] 10.222.0.89 () {58 vars in 1153 bytes} [Thu Jul 28 17:53:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:57.740135 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc33006f-2b1e-435a-87e0-eaf493c40420 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:57.751536 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc33006f-2b1e-435a-87e0-eaf493c40420 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:57.757731 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8722f9-6b9a-48cb-8578-6b2f64242bc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:57.758410 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8722f9-6b9a-48cb-8578-6b2f64242bc4 admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:57.758688 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8722f9-6b9a-48cb-8578-6b2f64242bc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:57.759019 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8722f9-6b9a-48cb-8578-6b2f64242bc4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:57.759688 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8722f9-6b9a-48cb-8578-6b2f64242bc4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:57.762536 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8722f9-6b9a-48cb-8578-6b2f64242bc4 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:57.773997 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-7a9b1463-6e8a-470f-a4b7-448dc93e504b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aedeb9e3e967402aa1458ceedf023dac'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:53:57.783728 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-7a9b1463-6e8a-470f-a4b7-448dc93e504b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aedeb9e3e967402aa1458ceedf023dac'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:53:57.795616 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-7a9b1463-6e8a-470f-a4b7-448dc93e504b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aedeb9e3e967402aa1458ceedf023dac'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:53:57.796453 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1351/2675] 10.222.0.89 () {60 vars in 1153 bytes} [Thu Jul 28 17:53:57 2022] POST /identity/v3/projects => generated 373 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:53:57.797307 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-7a9b1463-6e8a-470f-a4b7-448dc93e504b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aedeb9e3e967402aa1458ceedf023dac'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:53:57.800050 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-7a9b1463-6e8a-470f-a4b7-448dc93e504b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aedeb9e3e967402aa1458ceedf023dac'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:53:57.801141 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8722f9-6b9a-48cb-8578-6b2f64242bc4 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:57.804550 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-0e8722f9-6b9a-48cb-8578-6b2f64242bc4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:57.830974 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1325/2676] 10.222.0.89 () {58 vars in 1224 bytes} [Thu Jul 28 17:53:57 2022] DELETE /identity/v3/users/aedeb9e3e967402aa1458ceedf023dac => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:53:57.893016 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8722f9-6b9a-48cb-8578-6b2f64242bc4 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:57.976756 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9b1463-6e8a-470f-a4b7-448dc93e504b admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:57.978240 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-7a9b1463-6e8a-470f-a4b7-448dc93e504b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:57.982182 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffde7d6e-fc73-410a-95df-2c1609c24ece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:57.983615 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffde7d6e-fc73-410a-95df-2c1609c24ece admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:57.984089 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffde7d6e-fc73-410a-95df-2c1609c24ece admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:57.984369 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffde7d6e-fc73-410a-95df-2c1609c24ece admin admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:57.985042 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffde7d6e-fc73-410a-95df-2c1609c24ece admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:57.987924 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffde7d6e-fc73-410a-95df-2c1609c24ece admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:58.013403 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1352/2677] 10.222.0.89 () {60 vars in 1144 bytes} [Thu Jul 28 17:53:57 2022] POST /identity/v3/users => generated 287 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:53:58.017572 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffde7d6e-fc73-410a-95df-2c1609c24ece admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:58.030650 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffde7d6e-fc73-410a-95df-2c1609c24ece admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:58.039368 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f531114-929b-43b3-a8d9-dbfadf8f9e79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:58.039368 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f531114-929b-43b3-a8d9-dbfadf8f9e79 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:58.039368 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f531114-929b-43b3-a8d9-dbfadf8f9e79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:58.039368 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f531114-929b-43b3-a8d9-dbfadf8f9e79 admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:58.039804 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f531114-929b-43b3-a8d9-dbfadf8f9e79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:58.040804 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f531114-929b-43b3-a8d9-dbfadf8f9e79 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:58.057269 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1353/2678] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:53:58 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 17:53:58.057269 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f531114-929b-43b3-a8d9-dbfadf8f9e79 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:58.057743 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9b1463-6e8a-470f-a4b7-448dc93e504b admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:58.059701 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-3f531114-929b-43b3-a8d9-dbfadf8f9e79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:58.061540 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d8bbba-075c-47a9-93a1-7a2906edcc12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76817cd541294331b0023069092e8e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:58.062170 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d8bbba-075c-47a9-93a1-7a2906edcc12 service neutron] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:58.062453 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d8bbba-075c-47a9-93a1-7a2906edcc12 service neutron] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:58.062728 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d8bbba-075c-47a9-93a1-7a2906edcc12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:58.064947 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-a8d8bbba-075c-47a9-93a1-7a2906edcc12 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:58.077522 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d8bbba-075c-47a9-93a1-7a2906edcc12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:58.079767 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d8bbba-075c-47a9-93a1-7a2906edcc12 service neutron] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:58.082878 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f531114-929b-43b3-a8d9-dbfadf8f9e79 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:58.088217 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ff912f-c9f7-45d9-a4f0-b963308221fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:58.089998 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ff912f-c9f7-45d9-a4f0-b963308221fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:58.090152 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ff912f-c9f7-45d9-a4f0-b963308221fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:58.090314 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ff912f-c9f7-45d9-a4f0-b963308221fa admin admin] PATH_INFO: `/v3/projects/51f7a6e9080d48418e4912be02e09922` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:58.091535 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ff912f-c9f7-45d9-a4f0-b963308221fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=51f7a6e9080d48418e4912be02e09922)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:58.093345 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1326/2679] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:53:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 17:53:58.093494 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ff912f-c9f7-45d9-a4f0-b963308221fa admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:58.094493 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d8bbba-075c-47a9-93a1-7a2906edcc12 service neutron] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:58.110981 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d8bbba-075c-47a9-93a1-7a2906edcc12 service neutron] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:58.117881 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e049662b-96a9-4a44-b2dd-f8b6d96d0d6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:58.119178 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e049662b-96a9-4a44-b2dd-f8b6d96d0d6d admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:58.120078 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e049662b-96a9-4a44-b2dd-f8b6d96d0d6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:58.120938 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e049662b-96a9-4a44-b2dd-f8b6d96d0d6d admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:58.121849 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e049662b-96a9-4a44-b2dd-f8b6d96d0d6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:58.124436 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e049662b-96a9-4a44-b2dd-f8b6d96d0d6d admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:58.138435 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1327/2680] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:53:58 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:53:58.142354 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e049662b-96a9-4a44-b2dd-f8b6d96d0d6d admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:58.143290 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:INSERT=5,keystone:DELETE=3,keystone:SELECT=92 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:58.154591 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e049662b-96a9-4a44-b2dd-f8b6d96d0d6d admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:58.162417 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-12ff912f-c9f7-45d9-a4f0-b963308221fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '51f7a6e9080d48418e4912be02e09922', 'request_id': 'req-12ff912f-c9f7-45d9-a4f0-b963308221fa'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:53:58.169546 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-12ff912f-c9f7-45d9-a4f0-b963308221fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '51f7a6e9080d48418e4912be02e09922', 'request_id': 'req-12ff912f-c9f7-45d9-a4f0-b963308221fa'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:53:58.177400 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-12ff912f-c9f7-45d9-a4f0-b963308221fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '51f7a6e9080d48418e4912be02e09922', 'request_id': 'req-12ff912f-c9f7-45d9-a4f0-b963308221fa'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:53:58.181038 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:58.191666 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-12ff912f-c9f7-45d9-a4f0-b963308221fa admin admin] The token cache is being invalidate because project 51f7a6e9080d48418e4912be02e09922 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80324) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 17:53:58.191966 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-12ff912f-c9f7-45d9-a4f0-b963308221fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:53:58.192616 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-12ff912f-c9f7-45d9-a4f0-b963308221fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:53:58.195247 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1354/2681] 10.222.0.89 () {58 vars in 1233 bytes} [Thu Jul 28 17:53:58 2022] DELETE /identity/v3/projects/51f7a6e9080d48418e4912be02e09922 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:53:58.200959 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400bb68d-71dc-43b5-bbf9-9b55644d4891 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:58.201409 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400bb68d-71dc-43b5-bbf9-9b55644d4891 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:58.201813 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400bb68d-71dc-43b5-bbf9-9b55644d4891 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:58.222957 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-400bb68d-71dc-43b5-bbf9-9b55644d4891 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:53:58.224198 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-400bb68d-71dc-43b5-bbf9-9b55644d4891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:58.234753 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8268dea-fea2-4e6e-a2b2-81d19cae33b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:58.235687 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8268dea-fea2-4e6e-a2b2-81d19cae33b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:58.235964 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8268dea-fea2-4e6e-a2b2-81d19cae33b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:58.236234 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8268dea-fea2-4e6e-a2b2-81d19cae33b8 admin admin] PATH_INFO: `/v3/projects/9696eb4c822f429b8f90f4e99faf90fe/users/51be0c46e27443128ada43075a85e151/roles/11f8731756fe4caca13efa61b71b11c5` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:58.255706 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8268dea-fea2-4e6e-a2b2-81d19cae33b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9696eb4c822f429b8f90f4e99faf90fe, user_id=51be0c46e27443128ada43075a85e151, role_id=11f8731756fe4caca13efa61b71b11c5)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:58.256769 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8268dea-fea2-4e6e-a2b2-81d19cae33b8 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:58.274271 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1328/2682] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:53:58 2022] PUT /identity/v3/projects/9696eb4c822f429b8f90f4e99faf90fe/users/51be0c46e27443128ada43075a85e151/roles/11f8731756fe4caca13efa61b71b11c5 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 17:53:58.286280 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b987e8-8757-4b7b-a66a-657e5aef837e None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:58.286570 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b987e8-8757-4b7b-a66a-657e5aef837e None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:58.286848 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b987e8-8757-4b7b-a66a-657e5aef837e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:58.305062 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1355/2683] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:53:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 17:53:58.313099 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e323bd-3c3a-4bc9-a362-76ff97f06a05 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:58.313412 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e323bd-3c3a-4bc9-a362-76ff97f06a05 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:58.313686 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e323bd-3c3a-4bc9-a362-76ff97f06a05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:58.315788 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-20b987e8-8757-4b7b-a66a-657e5aef837e None None] MFA Rules not processed for user `51be0c46e27443128ada43075a85e151`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:53:58.317013 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-20b987e8-8757-4b7b-a66a-657e5aef837e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:58.336804 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-27e323bd-3c3a-4bc9-a362-76ff97f06a05 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:53:58.337821 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-27e323bd-3c3a-4bc9-a362-76ff97f06a05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:58.399881 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1329/2684] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:53:58 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 17:53:58.412171 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b987e8-8757-4b7b-a66a-657e5aef837e None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:58.413883 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-20b987e8-8757-4b7b-a66a-657e5aef837e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:58.419938 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1356/2685] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:53:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 17:53:58.422357 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e323bd-3c3a-4bc9-a362-76ff97f06a05 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:58.433079 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e323bd-3c3a-4bc9-a362-76ff97f06a05 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:58.438288 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c72704a-8c8e-4ed4-95f3-338b7c25d38e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:58.439183 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c72704a-8c8e-4ed4-95f3-338b7c25d38e admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:58.439566 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c72704a-8c8e-4ed4-95f3-338b7c25d38e admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:58.439938 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c72704a-8c8e-4ed4-95f3-338b7c25d38e admin admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:58.440738 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c72704a-8c8e-4ed4-95f3-338b7c25d38e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:53:58.441146 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c72704a-8c8e-4ed4-95f3-338b7c25d38e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:58.442902 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c72704a-8c8e-4ed4-95f3-338b7c25d38e admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:58.456677 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1357/2686] 10.222.0.89 () {58 vars in 1153 bytes} [Thu Jul 28 17:53:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:53:58.460601 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c72704a-8c8e-4ed4-95f3-338b7c25d38e admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:58.471480 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c72704a-8c8e-4ed4-95f3-338b7c25d38e admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:58.476749 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c457eea-1ca6-46d6-ab03-d76cbc1b2549 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:58.477356 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b987e8-8757-4b7b-a66a-657e5aef837e None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:58.477517 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c457eea-1ca6-46d6-ab03-d76cbc1b2549 admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:58.477693 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c457eea-1ca6-46d6-ab03-d76cbc1b2549 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:58.478334 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c457eea-1ca6-46d6-ab03-d76cbc1b2549 admin admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:58.478579 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c457eea-1ca6-46d6-ab03-d76cbc1b2549 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:58.480522 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c457eea-1ca6-46d6-ab03-d76cbc1b2549 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:58.481590 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da98eb7a-a487-4b19-b1df-5e2936eda482 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96d512d0c84240918dae6c4a3e8b7b79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:58.482210 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da98eb7a-a487-4b19-b1df-5e2936eda482 service cinder] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:58.482487 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da98eb7a-a487-4b19-b1df-5e2936eda482 service cinder] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:58.482745 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da98eb7a-a487-4b19-b1df-5e2936eda482 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:58.491702 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da98eb7a-a487-4b19-b1df-5e2936eda482 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:58.493316 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da98eb7a-a487-4b19-b1df-5e2936eda482 service cinder] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:58.503578 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1330/2687] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:53:58 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 17:53:58.521273 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1358/2688] 10.222.0.89 () {60 vars in 1153 bytes} [Thu Jul 28 17:53:58 2022] POST /identity/v3/projects => generated 375 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 17:53:58.524097 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da98eb7a-a487-4b19-b1df-5e2936eda482 service cinder] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:58.536911 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c457eea-1ca6-46d6-ab03-d76cbc1b2549 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:58.538882 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-9c457eea-1ca6-46d6-ab03-d76cbc1b2549 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:58.543181 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da98eb7a-a487-4b19-b1df-5e2936eda482 service cinder] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:58.568699 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:58.581550 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:58.607601 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e318b0-da2c-4128-8b74-81044e6ae18f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:58.608227 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e318b0-da2c-4128-8b74-81044e6ae18f admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:58.608541 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e318b0-da2c-4128-8b74-81044e6ae18f admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:58.608809 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e318b0-da2c-4128-8b74-81044e6ae18f admin admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:58.610004 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e318b0-da2c-4128-8b74-81044e6ae18f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:58.611953 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e318b0-da2c-4128-8b74-81044e6ae18f admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:58.617018 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c457eea-1ca6-46d6-ab03-d76cbc1b2549 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:58.623023 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f91cc3-c05d-4a40-8a7d-4b5a2b4231ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:58.623672 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f91cc3-c05d-4a40-8a7d-4b5a2b4231ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:58.623957 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f91cc3-c05d-4a40-8a7d-4b5a2b4231ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:58.624232 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f91cc3-c05d-4a40-8a7d-4b5a2b4231ca admin admin] PATH_INFO: `/v3/users/51be0c46e27443128ada43075a85e151` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:58.625452 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f91cc3-c05d-4a40-8a7d-4b5a2b4231ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=51be0c46e27443128ada43075a85e151)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:58.627434 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f91cc3-c05d-4a40-8a7d-4b5a2b4231ca admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:58.641946 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1331/2689] 10.222.0.89 () {60 vars in 1144 bytes} [Thu Jul 28 17:53:58 2022] POST /identity/v3/users => generated 288 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:53:58.643461 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e318b0-da2c-4128-8b74-81044e6ae18f admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:58.656411 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e318b0-da2c-4128-8b74-81044e6ae18f admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:58.661431 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f076ee89-9e21-4c6f-b81b-3e0621b3c10f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:58.662187 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f076ee89-9e21-4c6f-b81b-3e0621b3c10f admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:58.662573 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f076ee89-9e21-4c6f-b81b-3e0621b3c10f admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:58.662950 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f076ee89-9e21-4c6f-b81b-3e0621b3c10f admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:58.663752 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f076ee89-9e21-4c6f-b81b-3e0621b3c10f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:58.665603 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f076ee89-9e21-4c6f-b81b-3e0621b3c10f admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:58.678006 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1332/2690] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:53:58 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 17:53:58.681791 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f076ee89-9e21-4c6f-b81b-3e0621b3c10f admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:58.693333 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f076ee89-9e21-4c6f-b81b-3e0621b3c10f admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:58.694644 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-69f91cc3-c05d-4a40-8a7d-4b5a2b4231ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51be0c46e27443128ada43075a85e151'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:53:58.698944 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dd1883-1b26-4068-94ed-77cb2cb5c1b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:58.699888 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dd1883-1b26-4068-94ed-77cb2cb5c1b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:58.700175 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dd1883-1b26-4068-94ed-77cb2cb5c1b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:58.700574 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dd1883-1b26-4068-94ed-77cb2cb5c1b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:58.701144 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dd1883-1b26-4068-94ed-77cb2cb5c1b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:58.702776 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dd1883-1b26-4068-94ed-77cb2cb5c1b8 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:58.704071 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-69f91cc3-c05d-4a40-8a7d-4b5a2b4231ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51be0c46e27443128ada43075a85e151'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:53:58.704799 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-69f91cc3-c05d-4a40-8a7d-4b5a2b4231ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51be0c46e27443128ada43075a85e151'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:53:58.713903 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-69f91cc3-c05d-4a40-8a7d-4b5a2b4231ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51be0c46e27443128ada43075a85e151'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:53:58.715580 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1333/2691] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:53:58 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:53:58.719975 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dd1883-1b26-4068-94ed-77cb2cb5c1b8 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:58.722263 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-d8dd1883-1b26-4068-94ed-77cb2cb5c1b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:58.731142 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-69f91cc3-c05d-4a40-8a7d-4b5a2b4231ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51be0c46e27443128ada43075a85e151'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:53:58.739351 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1359/2692] 10.222.0.89 () {58 vars in 1224 bytes} [Thu Jul 28 17:53:58 2022] DELETE /identity/v3/users/51be0c46e27443128ada43075a85e151 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:53:58.739533 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dd1883-1b26-4068-94ed-77cb2cb5c1b8 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:58.743707 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec988afe-b9f1-43eb-b010-570c8194548b None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:58.744185 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec988afe-b9f1-43eb-b010-570c8194548b None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:58.744305 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec988afe-b9f1-43eb-b010-570c8194548b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:58.765275 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-ec988afe-b9f1-43eb-b010-570c8194548b None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:53:58.766499 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-ec988afe-b9f1-43eb-b010-570c8194548b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:58.819346 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81bc24d-4cbe-4aea-93f2-aed4eb10b174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:58.820000 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81bc24d-4cbe-4aea-93f2-aed4eb10b174 admin admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:58.820409 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81bc24d-4cbe-4aea-93f2-aed4eb10b174 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:58.820761 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81bc24d-4cbe-4aea-93f2-aed4eb10b174 admin admin] PATH_INFO: `/v3/projects/86318ba0c08b4934aaeb9c71aa3b3d86/users/a1bc2942abfb4edaace3286d2a1b33fd/roles/11f8731756fe4caca13efa61b71b11c5` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:58.839431 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81bc24d-4cbe-4aea-93f2-aed4eb10b174 admin admin] RBAC: Authorizing `identity:create_grant(project_id=86318ba0c08b4934aaeb9c71aa3b3d86, user_id=a1bc2942abfb4edaace3286d2a1b33fd, role_id=11f8731756fe4caca13efa61b71b11c5)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:58.842481 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81bc24d-4cbe-4aea-93f2-aed4eb10b174 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:58.847195 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1360/2693] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:53:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 17:53:58.857188 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec988afe-b9f1-43eb-b010-570c8194548b None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:58.858760 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-ec988afe-b9f1-43eb-b010-570c8194548b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:58.867554 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1334/2694] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:53:58 2022] PUT /identity/v3/projects/86318ba0c08b4934aaeb9c71aa3b3d86/users/a1bc2942abfb4edaace3286d2a1b33fd/roles/11f8731756fe4caca13efa61b71b11c5 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 17:53:58.871214 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965b0f74-5916-4951-9d71-7f68f3b01115 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:58.871615 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965b0f74-5916-4951-9d71-7f68f3b01115 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:58.871958 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965b0f74-5916-4951-9d71-7f68f3b01115 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:58.902846 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-965b0f74-5916-4951-9d71-7f68f3b01115 None None] MFA Rules not processed for user `a1bc2942abfb4edaace3286d2a1b33fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:53:58.904192 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-965b0f74-5916-4951-9d71-7f68f3b01115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:58.921129 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:58.936807 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec988afe-b9f1-43eb-b010-570c8194548b None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:58.983697 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1335/2695] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:53:58 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:53:58.996381 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965b0f74-5916-4951-9d71-7f68f3b01115 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:58.998133 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-965b0f74-5916-4951-9d71-7f68f3b01115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:59.000169 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c100f2a0-2941-4022-845b-33e111b86b87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76817cd541294331b0023069092e8e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:59.001125 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c100f2a0-2941-4022-845b-33e111b86b87 service neutron] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:59.001529 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c100f2a0-2941-4022-845b-33e111b86b87 service neutron] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:59.001898 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c100f2a0-2941-4022-845b-33e111b86b87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:59.026746 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:53:59.077445 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965b0f74-5916-4951-9d71-7f68f3b01115 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:59.078481 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c100f2a0-2941-4022-845b-33e111b86b87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:59.080109 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c100f2a0-2941-4022-845b-33e111b86b87 service neutron] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:59.081615 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0ee1e2-7b13-47e2-b87a-476178ee31de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96d512d0c84240918dae6c4a3e8b7b79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:59.082309 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0ee1e2-7b13-47e2-b87a-476178ee31de service cinder] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:59.082654 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0ee1e2-7b13-47e2-b87a-476178ee31de service cinder] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:59.083195 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0ee1e2-7b13-47e2-b87a-476178ee31de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:59.092121 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1361/2696] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:53:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 17:53:59.092703 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0ee1e2-7b13-47e2-b87a-476178ee31de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:59.094248 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0ee1e2-7b13-47e2-b87a-476178ee31de service cinder] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:59.105785 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1336/2697] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:53:58 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:53:59.565800 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c100f2a0-2941-4022-845b-33e111b86b87 service neutron] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:53:59.567919 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-c100f2a0-2941-4022-845b-33e111b86b87 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:53:59.584650 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c100f2a0-2941-4022-845b-33e111b86b87 service neutron] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:53:59.590618 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1ade94-7742-4200-82f2-fff234219a06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:53:59.591518 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1ade94-7742-4200-82f2-fff234219a06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:53:59.591817 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1ade94-7742-4200-82f2-fff234219a06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:53:59.593423 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1ade94-7742-4200-82f2-fff234219a06 admin admin] PATH_INFO: `/v3/projects/9696eb4c822f429b8f90f4e99faf90fe` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:53:59.595297 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1ade94-7742-4200-82f2-fff234219a06 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9696eb4c822f429b8f90f4e99faf90fe)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:53:59.597319 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1ade94-7742-4200-82f2-fff234219a06 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:53:59.656671 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-6e1ade94-7742-4200-82f2-fff234219a06 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9696eb4c822f429b8f90f4e99faf90fe', 'request_id': 'req-6e1ade94-7742-4200-82f2-fff234219a06'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:53:59.662808 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-6e1ade94-7742-4200-82f2-fff234219a06 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9696eb4c822f429b8f90f4e99faf90fe', 'request_id': 'req-6e1ade94-7742-4200-82f2-fff234219a06'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:53:59.669326 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-6e1ade94-7742-4200-82f2-fff234219a06 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9696eb4c822f429b8f90f4e99faf90fe', 'request_id': 'req-6e1ade94-7742-4200-82f2-fff234219a06'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:53:59.682074 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-6e1ade94-7742-4200-82f2-fff234219a06 admin admin] The token cache is being invalidate because project 9696eb4c822f429b8f90f4e99faf90fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80324) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 17:53:59.682895 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-6e1ade94-7742-4200-82f2-fff234219a06 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:53:59.683535 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-6e1ade94-7742-4200-82f2-fff234219a06 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:53:59.685077 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1362/2698] 10.222.0.89 () {58 vars in 1233 bytes} [Thu Jul 28 17:53:59 2022] DELETE /identity/v3/projects/9696eb4c822f429b8f90f4e99faf90fe => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:54:05.148231 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0ee1e2-7b13-47e2-b87a-476178ee31de service cinder] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:05.150530 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-ae0ee1e2-7b13-47e2-b87a-476178ee31de service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:54:05.178050 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0ee1e2-7b13-47e2-b87a-476178ee31de service cinder] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:05.250170 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189f61c6-4cc4-4879-b5bf-cca61c395d95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:05.250838 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189f61c6-4cc4-4879-b5bf-cca61c395d95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:05.251115 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189f61c6-4cc4-4879-b5bf-cca61c395d95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:05.251386 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189f61c6-4cc4-4879-b5bf-cca61c395d95 admin admin] PATH_INFO: `/v3/users/1cd17607cef54135a91f39ccd74b6180` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:05.252583 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189f61c6-4cc4-4879-b5bf-cca61c395d95 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1cd17607cef54135a91f39ccd74b6180)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:05.254509 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189f61c6-4cc4-4879-b5bf-cca61c395d95 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:05.317390 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-189f61c6-4cc4-4879-b5bf-cca61c395d95 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1cd17607cef54135a91f39ccd74b6180'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:54:05.317390 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-189f61c6-4cc4-4879-b5bf-cca61c395d95 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1cd17607cef54135a91f39ccd74b6180'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:54:05.318776 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-189f61c6-4cc4-4879-b5bf-cca61c395d95 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1cd17607cef54135a91f39ccd74b6180'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:54:05.319475 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-189f61c6-4cc4-4879-b5bf-cca61c395d95 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1cd17607cef54135a91f39ccd74b6180'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:54:05.320104 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-189f61c6-4cc4-4879-b5bf-cca61c395d95 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1cd17607cef54135a91f39ccd74b6180'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:54:05.340051 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1337/2699] 10.222.0.89 () {58 vars in 1224 bytes} [Thu Jul 28 17:54:05 2022] DELETE /identity/v3/users/1cd17607cef54135a91f39ccd74b6180 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:54:05.526151 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1ade94-7742-4200-82f2-fff234219a06 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:05.528712 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-6e1ade94-7742-4200-82f2-fff234219a06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:54:05.546749 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1ade94-7742-4200-82f2-fff234219a06 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:05.582813 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:05.617921 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec3e5d6-5219-425d-8944-5cae34c94828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:05.618752 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec3e5d6-5219-425d-8944-5cae34c94828 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:05.619105 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec3e5d6-5219-425d-8944-5cae34c94828 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:05.619443 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec3e5d6-5219-425d-8944-5cae34c94828 admin admin] PATH_INFO: `/v3/projects/0ef795b478944722886f1d084f49008d` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:05.622004 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec3e5d6-5219-425d-8944-5cae34c94828 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ef795b478944722886f1d084f49008d)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:05.624117 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec3e5d6-5219-425d-8944-5cae34c94828 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:05.676411 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-eec3e5d6-5219-425d-8944-5cae34c94828 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ef795b478944722886f1d084f49008d', 'request_id': 'req-eec3e5d6-5219-425d-8944-5cae34c94828'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:54:05.682621 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-eec3e5d6-5219-425d-8944-5cae34c94828 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ef795b478944722886f1d084f49008d', 'request_id': 'req-eec3e5d6-5219-425d-8944-5cae34c94828'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:54:05.691010 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-eec3e5d6-5219-425d-8944-5cae34c94828 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ef795b478944722886f1d084f49008d', 'request_id': 'req-eec3e5d6-5219-425d-8944-5cae34c94828'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:54:05.704723 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-eec3e5d6-5219-425d-8944-5cae34c94828 admin admin] The token cache is being invalidate because project 0ef795b478944722886f1d084f49008d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80324) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 17:54:05.705067 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-eec3e5d6-5219-425d-8944-5cae34c94828 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:54:05.705782 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-eec3e5d6-5219-425d-8944-5cae34c94828 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:54:05.707327 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1363/2700] 10.222.0.89 () {58 vars in 1233 bytes} [Thu Jul 28 17:54:05 2022] DELETE /identity/v3/projects/0ef795b478944722886f1d084f49008d => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:54:05.711501 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189f61c6-4cc4-4879-b5bf-cca61c395d95 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:05.713028 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-189f61c6-4cc4-4879-b5bf-cca61c395d95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:54:05.728087 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189f61c6-4cc4-4879-b5bf-cca61c395d95 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:05.773603 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:05.796386 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952db0ec-c6c1-48bf-9ff1-919f0bff4576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:05.797048 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952db0ec-c6c1-48bf-9ff1-919f0bff4576 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:05.797361 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952db0ec-c6c1-48bf-9ff1-919f0bff4576 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:05.797647 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952db0ec-c6c1-48bf-9ff1-919f0bff4576 admin admin] PATH_INFO: `/v3/users/7f925222c6f04622b5ee22c10f0ba48b` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:05.798809 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952db0ec-c6c1-48bf-9ff1-919f0bff4576 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f925222c6f04622b5ee22c10f0ba48b)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:05.800710 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952db0ec-c6c1-48bf-9ff1-919f0bff4576 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:05.855695 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-952db0ec-c6c1-48bf-9ff1-919f0bff4576 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f925222c6f04622b5ee22c10f0ba48b'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:54:05.862913 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-952db0ec-c6c1-48bf-9ff1-919f0bff4576 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f925222c6f04622b5ee22c10f0ba48b'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:54:05.870051 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-952db0ec-c6c1-48bf-9ff1-919f0bff4576 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f925222c6f04622b5ee22c10f0ba48b'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:54:05.870832 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-952db0ec-c6c1-48bf-9ff1-919f0bff4576 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f925222c6f04622b5ee22c10f0ba48b'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:54:05.871444 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-952db0ec-c6c1-48bf-9ff1-919f0bff4576 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f925222c6f04622b5ee22c10f0ba48b'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:54:05.891576 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1338/2701] 10.222.0.89 () {58 vars in 1224 bytes} [Thu Jul 28 17:54:05 2022] DELETE /identity/v3/users/7f925222c6f04622b5ee22c10f0ba48b => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 17:54:06.079319 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec3e5d6-5219-425d-8944-5cae34c94828 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:06.080811 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-eec3e5d6-5219-425d-8944-5cae34c94828 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:54:06.094423 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec3e5d6-5219-425d-8944-5cae34c94828 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:06.173303 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e19fec-973c-4b28-99e9-ed45d6f17ea0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:06.174512 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e19fec-973c-4b28-99e9-ed45d6f17ea0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:06.175149 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e19fec-973c-4b28-99e9-ed45d6f17ea0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:06.175556 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e19fec-973c-4b28-99e9-ed45d6f17ea0 admin admin] PATH_INFO: `/v3/projects/54265825d5534d4b9f49e5633615db1d` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:06.177557 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e19fec-973c-4b28-99e9-ed45d6f17ea0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=54265825d5534d4b9f49e5633615db1d)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:06.180804 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e19fec-973c-4b28-99e9-ed45d6f17ea0 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:06.253303 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-e9e19fec-973c-4b28-99e9-ed45d6f17ea0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '54265825d5534d4b9f49e5633615db1d', 'request_id': 'req-e9e19fec-973c-4b28-99e9-ed45d6f17ea0'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:54:06.262171 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-e9e19fec-973c-4b28-99e9-ed45d6f17ea0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '54265825d5534d4b9f49e5633615db1d', 'request_id': 'req-e9e19fec-973c-4b28-99e9-ed45d6f17ea0'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:54:06.268832 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-e9e19fec-973c-4b28-99e9-ed45d6f17ea0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '54265825d5534d4b9f49e5633615db1d', 'request_id': 'req-e9e19fec-973c-4b28-99e9-ed45d6f17ea0'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:54:06.271421 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:06.283901 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-e9e19fec-973c-4b28-99e9-ed45d6f17ea0 admin admin] The token cache is being invalidate because project 54265825d5534d4b9f49e5633615db1d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80324) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 17:54:06.284186 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-e9e19fec-973c-4b28-99e9-ed45d6f17ea0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:54:06.284816 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-e9e19fec-973c-4b28-99e9-ed45d6f17ea0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:54:06.286361 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1364/2702] 10.222.0.89 () {58 vars in 1233 bytes} [Thu Jul 28 17:54:06 2022] DELETE /identity/v3/projects/54265825d5534d4b9f49e5633615db1d => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 17:54:09.554425 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952db0ec-c6c1-48bf-9ff1-919f0bff4576 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:09.555821 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-952db0ec-c6c1-48bf-9ff1-919f0bff4576 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:54:09.569439 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952db0ec-c6c1-48bf-9ff1-919f0bff4576 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:09.625558 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace125bd-e96b-4f2e-9ead-d7451b71a80e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:09.626200 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace125bd-e96b-4f2e-9ead-d7451b71a80e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:09.626488 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace125bd-e96b-4f2e-9ead-d7451b71a80e admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:09.626763 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace125bd-e96b-4f2e-9ead-d7451b71a80e admin admin] PATH_INFO: `/v3/users/a1bc2942abfb4edaace3286d2a1b33fd` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:09.628184 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace125bd-e96b-4f2e-9ead-d7451b71a80e admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1bc2942abfb4edaace3286d2a1b33fd)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:09.630172 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace125bd-e96b-4f2e-9ead-d7451b71a80e admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:09.685925 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-ace125bd-e96b-4f2e-9ead-d7451b71a80e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1bc2942abfb4edaace3286d2a1b33fd'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:54:09.692750 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-ace125bd-e96b-4f2e-9ead-d7451b71a80e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1bc2942abfb4edaace3286d2a1b33fd'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:54:09.696394 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:09.700533 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-ace125bd-e96b-4f2e-9ead-d7451b71a80e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1bc2942abfb4edaace3286d2a1b33fd'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:54:09.701409 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-ace125bd-e96b-4f2e-9ead-d7451b71a80e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1bc2942abfb4edaace3286d2a1b33fd'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:54:09.702236 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-ace125bd-e96b-4f2e-9ead-d7451b71a80e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1bc2942abfb4edaace3286d2a1b33fd'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:54:09.721027 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1339/2703] 10.222.0.89 () {58 vars in 1224 bytes} [Thu Jul 28 17:54:09 2022] DELETE /identity/v3/users/a1bc2942abfb4edaace3286d2a1b33fd => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:54:09.725759 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4870178e-a064-470e-a7ea-70990d03776f None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:09.726246 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4870178e-a064-470e-a7ea-70990d03776f None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:09.726364 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4870178e-a064-470e-a7ea-70990d03776f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:09.746269 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-4870178e-a064-470e-a7ea-70990d03776f None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:54:09.747762 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-4870178e-a064-470e-a7ea-70990d03776f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:54:09.807744 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1365/2704] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:54:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:54:09.817549 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace125bd-e96b-4f2e-9ead-d7451b71a80e admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:09.818971 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-ace125bd-e96b-4f2e-9ead-d7451b71a80e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:54:09.877598 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace125bd-e96b-4f2e-9ead-d7451b71a80e admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:09.881377 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d347f51b-1e91-468d-8f86-c04182424953 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76817cd541294331b0023069092e8e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:09.881991 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d347f51b-1e91-468d-8f86-c04182424953 service neutron] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:09.882272 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d347f51b-1e91-468d-8f86-c04182424953 service neutron] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:09.882547 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d347f51b-1e91-468d-8f86-c04182424953 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:09.892920 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d347f51b-1e91-468d-8f86-c04182424953 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:09.894380 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d347f51b-1e91-468d-8f86-c04182424953 service neutron] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:09.904303 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1340/2705] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:54:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:54:10.185510 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4870178e-a064-470e-a7ea-70990d03776f None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:10.187057 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-4870178e-a064-470e-a7ea-70990d03776f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:54:10.201235 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4870178e-a064-470e-a7ea-70990d03776f None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:10.205863 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737a3459-a746-4c79-a875-f8fd744115b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:10.206505 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737a3459-a746-4c79-a875-f8fd744115b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:10.206782 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737a3459-a746-4c79-a875-f8fd744115b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:10.207037 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737a3459-a746-4c79-a875-f8fd744115b3 admin admin] PATH_INFO: `/v3/projects/86318ba0c08b4934aaeb9c71aa3b3d86` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:10.208211 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737a3459-a746-4c79-a875-f8fd744115b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=86318ba0c08b4934aaeb9c71aa3b3d86)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:10.210333 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737a3459-a746-4c79-a875-f8fd744115b3 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:10.259823 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-737a3459-a746-4c79-a875-f8fd744115b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86318ba0c08b4934aaeb9c71aa3b3d86', 'request_id': 'req-737a3459-a746-4c79-a875-f8fd744115b3'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:54:10.266220 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-737a3459-a746-4c79-a875-f8fd744115b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86318ba0c08b4934aaeb9c71aa3b3d86', 'request_id': 'req-737a3459-a746-4c79-a875-f8fd744115b3'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:54:10.272330 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-737a3459-a746-4c79-a875-f8fd744115b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86318ba0c08b4934aaeb9c71aa3b3d86', 'request_id': 'req-737a3459-a746-4c79-a875-f8fd744115b3'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:54:10.285475 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-737a3459-a746-4c79-a875-f8fd744115b3 admin admin] The token cache is being invalidate because project 86318ba0c08b4934aaeb9c71aa3b3d86 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80324) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 17:54:10.285768 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-737a3459-a746-4c79-a875-f8fd744115b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:54:10.286397 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-737a3459-a746-4c79-a875-f8fd744115b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:54:10.287909 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1366/2706] 10.222.0.89 () {58 vars in 1233 bytes} [Thu Jul 28 17:54:10 2022] DELETE /identity/v3/projects/86318ba0c08b4934aaeb9c71aa3b3d86 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:54:10.293334 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26f67e6-1375-4597-9a04-50beb010f8e9 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:10.293626 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26f67e6-1375-4597-9a04-50beb010f8e9 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:10.293905 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26f67e6-1375-4597-9a04-50beb010f8e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:10.313842 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-f26f67e6-1375-4597-9a04-50beb010f8e9 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:54:10.315008 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-f26f67e6-1375-4597-9a04-50beb010f8e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:54:10.374479 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1341/2707] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:54:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 17:54:10.382152 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0635be3b-3d97-4642-b268-17591730a067 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:10.382533 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0635be3b-3d97-4642-b268-17591730a067 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:10.382817 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0635be3b-3d97-4642-b268-17591730a067 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:10.402727 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-0635be3b-3d97-4642-b268-17591730a067 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:54:10.403920 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-0635be3b-3d97-4642-b268-17591730a067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:54:10.411849 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1367/2708] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:54:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:54:10.416136 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26f67e6-1375-4597-9a04-50beb010f8e9 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:10.426514 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26f67e6-1375-4597-9a04-50beb010f8e9 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:10.431101 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097513e8-298a-4e0a-ba97-e9f1a39bdb2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:10.431710 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097513e8-298a-4e0a-ba97-e9f1a39bdb2b admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:10.431994 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097513e8-298a-4e0a-ba97-e9f1a39bdb2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:10.432270 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097513e8-298a-4e0a-ba97-e9f1a39bdb2b admin admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:10.433001 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097513e8-298a-4e0a-ba97-e9f1a39bdb2b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:54:10.433903 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097513e8-298a-4e0a-ba97-e9f1a39bdb2b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:10.435521 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097513e8-298a-4e0a-ba97-e9f1a39bdb2b admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:10.446947 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1342/2709] 10.222.0.89 () {58 vars in 1153 bytes} [Thu Jul 28 17:54:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:54:10.451256 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0635be3b-3d97-4642-b268-17591730a067 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:10.460661 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0635be3b-3d97-4642-b268-17591730a067 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:10.465662 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a327e1fa-b3bc-41ad-a398-64769a32f8c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:10.466278 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a327e1fa-b3bc-41ad-a398-64769a32f8c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:10.466588 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a327e1fa-b3bc-41ad-a398-64769a32f8c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:10.466860 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a327e1fa-b3bc-41ad-a398-64769a32f8c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:10.467570 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a327e1fa-b3bc-41ad-a398-64769a32f8c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:10.469517 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a327e1fa-b3bc-41ad-a398-64769a32f8c1 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:10.497678 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1368/2710] 10.222.0.89 () {60 vars in 1153 bytes} [Thu Jul 28 17:54:10 2022] POST /identity/v3/projects => generated 391 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:54:10.502074 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097513e8-298a-4e0a-ba97-e9f1a39bdb2b admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:10.512127 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097513e8-298a-4e0a-ba97-e9f1a39bdb2b admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:10.568975 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff41384f-efb6-49ce-b064-c1eed6ca12d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:10.569668 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff41384f-efb6-49ce-b064-c1eed6ca12d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:10.569821 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff41384f-efb6-49ce-b064-c1eed6ca12d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:10.570097 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff41384f-efb6-49ce-b064-c1eed6ca12d4 admin admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:10.570774 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff41384f-efb6-49ce-b064-c1eed6ca12d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:10.572679 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff41384f-efb6-49ce-b064-c1eed6ca12d4 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:10.590641 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:10.600977 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1343/2711] 10.222.0.89 () {60 vars in 1144 bytes} [Thu Jul 28 17:54:10 2022] POST /identity/v3/users => generated 296 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:54:10.602175 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a327e1fa-b3bc-41ad-a398-64769a32f8c1 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:10.612662 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a327e1fa-b3bc-41ad-a398-64769a32f8c1 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:10.617377 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424a39a2-905d-4fb0-bdc6-5c36b1bb8bc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:10.617974 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424a39a2-905d-4fb0-bdc6-5c36b1bb8bc2 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:10.618253 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424a39a2-905d-4fb0-bdc6-5c36b1bb8bc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:10.618529 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424a39a2-905d-4fb0-bdc6-5c36b1bb8bc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:10.619215 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424a39a2-905d-4fb0-bdc6-5c36b1bb8bc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:10.621313 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424a39a2-905d-4fb0-bdc6-5c36b1bb8bc2 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:10.631449 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1369/2712] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:54:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:54:10.635874 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff41384f-efb6-49ce-b064-c1eed6ca12d4 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:10.648388 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff41384f-efb6-49ce-b064-c1eed6ca12d4 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:10.653834 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7861e47b-ad7b-4f8f-aab0-c7c0b17d3cb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:10.654441 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7861e47b-ad7b-4f8f-aab0-c7c0b17d3cb4 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:10.654717 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7861e47b-ad7b-4f8f-aab0-c7c0b17d3cb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:10.654989 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7861e47b-ad7b-4f8f-aab0-c7c0b17d3cb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:10.655672 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7861e47b-ad7b-4f8f-aab0-c7c0b17d3cb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:10.657386 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7861e47b-ad7b-4f8f-aab0-c7c0b17d3cb4 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:10.669412 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1344/2713] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:54:10 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:54:10.673663 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424a39a2-905d-4fb0-bdc6-5c36b1bb8bc2 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:10.684930 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424a39a2-905d-4fb0-bdc6-5c36b1bb8bc2 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:10.689623 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761374cf-c2a6-4585-87f0-85aba585adaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:10.690233 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761374cf-c2a6-4585-87f0-85aba585adaf admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:10.690515 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761374cf-c2a6-4585-87f0-85aba585adaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:10.690784 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761374cf-c2a6-4585-87f0-85aba585adaf admin admin] PATH_INFO: `/v3/projects/5646310737ed4d42aa572e4c6cc1bf57/users/5316cbe17f4f4c75996127e3eab30fb8/roles/11f8731756fe4caca13efa61b71b11c5` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:10.705681 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761374cf-c2a6-4585-87f0-85aba585adaf admin admin] RBAC: Authorizing `identity:create_grant(project_id=5646310737ed4d42aa572e4c6cc1bf57, user_id=5316cbe17f4f4c75996127e3eab30fb8, role_id=11f8731756fe4caca13efa61b71b11c5)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:10.708333 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761374cf-c2a6-4585-87f0-85aba585adaf admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:10.721444 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1370/2714] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:54:10 2022] PUT /identity/v3/projects/5646310737ed4d42aa572e4c6cc1bf57/users/5316cbe17f4f4c75996127e3eab30fb8/roles/11f8731756fe4caca13efa61b71b11c5 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:54:10.731436 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b7b0b7-78af-45be-9832-63cb70bb46d8 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:10.731715 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b7b0b7-78af-45be-9832-63cb70bb46d8 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:10.731989 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b7b0b7-78af-45be-9832-63cb70bb46d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:10.759770 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-90b7b0b7-78af-45be-9832-63cb70bb46d8 None None] MFA Rules not processed for user `5316cbe17f4f4c75996127e3eab30fb8`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:54:10.761012 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-90b7b0b7-78af-45be-9832-63cb70bb46d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:54:10.825618 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1345/2715] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:54:10 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 17:54:10.836329 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761374cf-c2a6-4585-87f0-85aba585adaf admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:10.838081 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-761374cf-c2a6-4585-87f0-85aba585adaf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:54:10.852494 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:DELETE=8,keystone:SELECT=89,keystone:INSERT=3 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:10.898029 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761374cf-c2a6-4585-87f0-85aba585adaf admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:10.901768 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c53e3a2-cfe2-45a6-8f1a-aeebc1d94dfd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96d512d0c84240918dae6c4a3e8b7b79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:10.902691 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c53e3a2-cfe2-45a6-8f1a-aeebc1d94dfd service cinder] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:10.902975 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c53e3a2-cfe2-45a6-8f1a-aeebc1d94dfd service cinder] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:10.903253 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c53e3a2-cfe2-45a6-8f1a-aeebc1d94dfd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:10.911863 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c53e3a2-cfe2-45a6-8f1a-aeebc1d94dfd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:10.913321 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c53e3a2-cfe2-45a6-8f1a-aeebc1d94dfd service cinder] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:10.922886 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1371/2716] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:54:10 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:54:20.821933 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=38 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:20.918816 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=20 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:30.256046 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b7b0b7-78af-45be-9832-63cb70bb46d8 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:30.319612 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b7b0b7-78af-45be-9832-63cb70bb46d8 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:30.324252 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df77157-e12d-42a1-9b6f-83a985f39212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:30.324882 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df77157-e12d-42a1-9b6f-83a985f39212 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:30.325181 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df77157-e12d-42a1-9b6f-83a985f39212 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:30.325471 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df77157-e12d-42a1-9b6f-83a985f39212 admin admin] PATH_INFO: `/v3/users/5316cbe17f4f4c75996127e3eab30fb8` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:30.326637 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df77157-e12d-42a1-9b6f-83a985f39212 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5316cbe17f4f4c75996127e3eab30fb8)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:30.328580 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df77157-e12d-42a1-9b6f-83a985f39212 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:30.377735 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-6df77157-e12d-42a1-9b6f-83a985f39212 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5316cbe17f4f4c75996127e3eab30fb8'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:54:30.384568 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-6df77157-e12d-42a1-9b6f-83a985f39212 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5316cbe17f4f4c75996127e3eab30fb8'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:54:30.391917 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-6df77157-e12d-42a1-9b6f-83a985f39212 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5316cbe17f4f4c75996127e3eab30fb8'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:54:30.392606 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-6df77157-e12d-42a1-9b6f-83a985f39212 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5316cbe17f4f4c75996127e3eab30fb8'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:54:30.393254 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-6df77157-e12d-42a1-9b6f-83a985f39212 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5316cbe17f4f4c75996127e3eab30fb8'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:54:30.410838 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1346/2717] 10.222.0.89 () {58 vars in 1224 bytes} [Thu Jul 28 17:54:30 2022] DELETE /identity/v3/users/5316cbe17f4f4c75996127e3eab30fb8 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 17:54:30.415314 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0da92c5-c839-482c-b028-6253e6608126 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:30.415810 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0da92c5-c839-482c-b028-6253e6608126 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:30.415954 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0da92c5-c839-482c-b028-6253e6608126 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:30.435405 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-f0da92c5-c839-482c-b028-6253e6608126 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:54:30.436626 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-f0da92c5-c839-482c-b028-6253e6608126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:54:30.495301 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1372/2718] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:54:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:54:30.505780 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df77157-e12d-42a1-9b6f-83a985f39212 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:30.507230 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-6df77157-e12d-42a1-9b6f-83a985f39212 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:54:30.564190 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df77157-e12d-42a1-9b6f-83a985f39212 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:30.568062 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c4e8dc-e807-4b89-a6c8-26137693b380 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76817cd541294331b0023069092e8e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:30.568661 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c4e8dc-e807-4b89-a6c8-26137693b380 service neutron] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:30.568934 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c4e8dc-e807-4b89-a6c8-26137693b380 service neutron] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:30.569229 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c4e8dc-e807-4b89-a6c8-26137693b380 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:30.578070 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c4e8dc-e807-4b89-a6c8-26137693b380 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:30.579542 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c4e8dc-e807-4b89-a6c8-26137693b380 service neutron] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:30.588981 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1347/2719] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:54:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:54:30.857935 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0da92c5-c839-482c-b028-6253e6608126 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:30.859437 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-f0da92c5-c839-482c-b028-6253e6608126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:54:30.872197 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0da92c5-c839-482c-b028-6253e6608126 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:30.877769 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fbe5bc-9e32-4045-9510-9dc5eab9199e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:30.878416 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fbe5bc-9e32-4045-9510-9dc5eab9199e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:30.878692 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fbe5bc-9e32-4045-9510-9dc5eab9199e admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:30.878966 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fbe5bc-9e32-4045-9510-9dc5eab9199e admin admin] PATH_INFO: `/v3/projects/5646310737ed4d42aa572e4c6cc1bf57` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:30.881227 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fbe5bc-9e32-4045-9510-9dc5eab9199e admin admin] RBAC: Authorizing `identity:delete_project(project_id=5646310737ed4d42aa572e4c6cc1bf57)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:30.882094 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fbe5bc-9e32-4045-9510-9dc5eab9199e admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:30.933741 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-e1fbe5bc-9e32-4045-9510-9dc5eab9199e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5646310737ed4d42aa572e4c6cc1bf57', 'request_id': 'req-e1fbe5bc-9e32-4045-9510-9dc5eab9199e'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:54:30.939955 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-e1fbe5bc-9e32-4045-9510-9dc5eab9199e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5646310737ed4d42aa572e4c6cc1bf57', 'request_id': 'req-e1fbe5bc-9e32-4045-9510-9dc5eab9199e'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:54:30.947005 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-e1fbe5bc-9e32-4045-9510-9dc5eab9199e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5646310737ed4d42aa572e4c6cc1bf57', 'request_id': 'req-e1fbe5bc-9e32-4045-9510-9dc5eab9199e'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:54:30.962690 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-e1fbe5bc-9e32-4045-9510-9dc5eab9199e admin admin] The token cache is being invalidate because project 5646310737ed4d42aa572e4c6cc1bf57 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80324) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 17:54:30.962978 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-e1fbe5bc-9e32-4045-9510-9dc5eab9199e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:54:30.963621 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-e1fbe5bc-9e32-4045-9510-9dc5eab9199e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:54:30.965196 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1373/2720] 10.222.0.89 () {58 vars in 1233 bytes} [Thu Jul 28 17:54:30 2022] DELETE /identity/v3/projects/5646310737ed4d42aa572e4c6cc1bf57 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:54:30.970720 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8902cf-d0b6-4530-a76c-1ce129d4d298 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:30.971007 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8902cf-d0b6-4530-a76c-1ce129d4d298 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:30.971281 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8902cf-d0b6-4530-a76c-1ce129d4d298 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:30.991131 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-4b8902cf-d0b6-4530-a76c-1ce129d4d298 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:54:30.992337 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-4b8902cf-d0b6-4530-a76c-1ce129d4d298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:54:31.009673 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:31.053289 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1348/2721] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:54:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 17:54:31.060977 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad107ae-f124-488e-93cc-ad6c75928581 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:31.061305 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad107ae-f124-488e-93cc-ad6c75928581 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:31.061610 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad107ae-f124-488e-93cc-ad6c75928581 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:31.081877 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-3ad107ae-f124-488e-93cc-ad6c75928581 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:54:31.083117 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-3ad107ae-f124-488e-93cc-ad6c75928581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:54:31.090989 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1374/2722] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:54:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:54:31.095294 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8902cf-d0b6-4530-a76c-1ce129d4d298 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:31.105043 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8902cf-d0b6-4530-a76c-1ce129d4d298 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:31.110565 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ed8ae5-328f-4d65-9607-81fa32ee6730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:31.111474 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ed8ae5-328f-4d65-9607-81fa32ee6730 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:31.111754 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ed8ae5-328f-4d65-9607-81fa32ee6730 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:31.112036 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ed8ae5-328f-4d65-9607-81fa32ee6730 admin admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:31.112759 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ed8ae5-328f-4d65-9607-81fa32ee6730 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:54:31.113068 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ed8ae5-328f-4d65-9607-81fa32ee6730 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:31.114833 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ed8ae5-328f-4d65-9607-81fa32ee6730 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:31.127784 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1349/2723] 10.222.0.89 () {58 vars in 1153 bytes} [Thu Jul 28 17:54:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:54:31.132358 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad107ae-f124-488e-93cc-ad6c75928581 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:31.142046 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad107ae-f124-488e-93cc-ad6c75928581 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:31.146758 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9860037c-bd50-4260-8556-232b6cc9f322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:31.147354 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9860037c-bd50-4260-8556-232b6cc9f322 admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:31.147635 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9860037c-bd50-4260-8556-232b6cc9f322 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:31.147909 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9860037c-bd50-4260-8556-232b6cc9f322 admin admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:31.148566 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9860037c-bd50-4260-8556-232b6cc9f322 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:31.150483 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9860037c-bd50-4260-8556-232b6cc9f322 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:31.188408 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1375/2724] 10.222.0.89 () {60 vars in 1153 bytes} [Thu Jul 28 17:54:31 2022] POST /identity/v3/projects => generated 379 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:54:31.192683 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ed8ae5-328f-4d65-9607-81fa32ee6730 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:31.202733 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ed8ae5-328f-4d65-9607-81fa32ee6730 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:31.258669 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f55b22-9a0d-4deb-9e0c-2afb8dd65d91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:31.259348 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f55b22-9a0d-4deb-9e0c-2afb8dd65d91 admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:31.259556 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f55b22-9a0d-4deb-9e0c-2afb8dd65d91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:31.259830 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f55b22-9a0d-4deb-9e0c-2afb8dd65d91 admin admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:31.260508 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f55b22-9a0d-4deb-9e0c-2afb8dd65d91 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:31.262445 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f55b22-9a0d-4deb-9e0c-2afb8dd65d91 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:31.284772 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1350/2725] 10.222.0.89 () {60 vars in 1144 bytes} [Thu Jul 28 17:54:31 2022] POST /identity/v3/users => generated 290 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:54:31.289141 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9860037c-bd50-4260-8556-232b6cc9f322 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:31.300267 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9860037c-bd50-4260-8556-232b6cc9f322 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:31.305130 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cec8a5-b9e7-4cd4-a39b-d41b91940b84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:31.305765 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cec8a5-b9e7-4cd4-a39b-d41b91940b84 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:31.306050 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cec8a5-b9e7-4cd4-a39b-d41b91940b84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:31.306377 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cec8a5-b9e7-4cd4-a39b-d41b91940b84 admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:31.307153 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cec8a5-b9e7-4cd4-a39b-d41b91940b84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:31.308759 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cec8a5-b9e7-4cd4-a39b-d41b91940b84 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:31.319541 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1376/2726] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:54:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:54:31.323580 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f55b22-9a0d-4deb-9e0c-2afb8dd65d91 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:31.334261 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f55b22-9a0d-4deb-9e0c-2afb8dd65d91 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:31.338922 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00f6260-9b00-4e97-a6d9-8dc5f40d3f13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:31.339529 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00f6260-9b00-4e97-a6d9-8dc5f40d3f13 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:31.339811 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00f6260-9b00-4e97-a6d9-8dc5f40d3f13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:31.340084 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00f6260-9b00-4e97-a6d9-8dc5f40d3f13 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:31.340779 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00f6260-9b00-4e97-a6d9-8dc5f40d3f13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:31.342402 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00f6260-9b00-4e97-a6d9-8dc5f40d3f13 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:31.355675 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1351/2727] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:54:31 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 17:54:31.360220 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cec8a5-b9e7-4cd4-a39b-d41b91940b84 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:31.370153 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cec8a5-b9e7-4cd4-a39b-d41b91940b84 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:31.374731 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3f4a4a-feb7-4d0a-855d-5f0f85e6f798 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:31.376401 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3f4a4a-feb7-4d0a-855d-5f0f85e6f798 admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:31.376683 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3f4a4a-feb7-4d0a-855d-5f0f85e6f798 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:31.376958 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3f4a4a-feb7-4d0a-855d-5f0f85e6f798 admin admin] PATH_INFO: `/v3/projects/1f721aa2c35842beba2060fde666b2a1/users/c4618ca07ce2408ba90e9eaab0b940c9/roles/11f8731756fe4caca13efa61b71b11c5` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:31.393232 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3f4a4a-feb7-4d0a-855d-5f0f85e6f798 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f721aa2c35842beba2060fde666b2a1, user_id=c4618ca07ce2408ba90e9eaab0b940c9, role_id=11f8731756fe4caca13efa61b71b11c5)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:31.395931 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3f4a4a-feb7-4d0a-855d-5f0f85e6f798 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:31.409152 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1377/2728] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:54:31 2022] PUT /identity/v3/projects/1f721aa2c35842beba2060fde666b2a1/users/c4618ca07ce2408ba90e9eaab0b940c9/roles/11f8731756fe4caca13efa61b71b11c5 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:54:31.414152 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d52172-0dba-4562-b518-fb8497cbdf67 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:31.414440 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d52172-0dba-4562-b518-fb8497cbdf67 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:31.414718 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d52172-0dba-4562-b518-fb8497cbdf67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:31.436969 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-34d52172-0dba-4562-b518-fb8497cbdf67 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:54:31.438946 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-34d52172-0dba-4562-b518-fb8497cbdf67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:54:31.498601 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1352/2729] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:54:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:54:31.508991 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3f4a4a-feb7-4d0a-855d-5f0f85e6f798 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:31.510740 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-0d3f4a4a-feb7-4d0a-855d-5f0f85e6f798 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:54:31.524886 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:31.571328 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3f4a4a-feb7-4d0a-855d-5f0f85e6f798 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:31.575123 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9838ba-122a-48a7-b2fe-8e41a78df40c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76817cd541294331b0023069092e8e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:31.575759 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9838ba-122a-48a7-b2fe-8e41a78df40c service neutron] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:31.576038 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9838ba-122a-48a7-b2fe-8e41a78df40c service neutron] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:31.576308 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9838ba-122a-48a7-b2fe-8e41a78df40c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:31.586507 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9838ba-122a-48a7-b2fe-8e41a78df40c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:31.587977 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9838ba-122a-48a7-b2fe-8e41a78df40c service neutron] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:31.597862 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1378/2730] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:54:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:54:35.387749 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcab17e-4a9c-4df0-941f-f2503889396a None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:35.388321 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcab17e-4a9c-4df0-941f-f2503889396a None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:35.388321 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcab17e-4a9c-4df0-941f-f2503889396a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:35.417557 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-bbcab17e-4a9c-4df0-941f-f2503889396a None None] MFA Rules not processed for user `c4618ca07ce2408ba90e9eaab0b940c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:54:35.418762 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-bbcab17e-4a9c-4df0-941f-f2503889396a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:54:35.436375 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:35.488250 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1353/2731] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:54:35 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:54:35.492867 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9838ba-122a-48a7-b2fe-8e41a78df40c service neutron] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:35.503075 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9838ba-122a-48a7-b2fe-8e41a78df40c service neutron] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:35.507719 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b514ac0a-9a94-49e0-80fb-cfc674a2d98e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:35.508323 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b514ac0a-9a94-49e0-80fb-cfc674a2d98e admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:35.508600 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b514ac0a-9a94-49e0-80fb-cfc674a2d98e admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:35.508872 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b514ac0a-9a94-49e0-80fb-cfc674a2d98e admin admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:35.509564 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b514ac0a-9a94-49e0-80fb-cfc674a2d98e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:35.511553 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b514ac0a-9a94-49e0-80fb-cfc674a2d98e admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:35.539471 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1379/2732] 10.222.0.89 () {60 vars in 1153 bytes} [Thu Jul 28 17:54:35 2022] POST /identity/v3/projects => generated 379 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:54:35.542947 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcab17e-4a9c-4df0-941f-f2503889396a None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:35.554919 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcab17e-4a9c-4df0-941f-f2503889396a None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:35.615472 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c935ce9-6567-491f-bf49-b22cd034a663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:35.616096 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c935ce9-6567-491f-bf49-b22cd034a663 admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:35.616373 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c935ce9-6567-491f-bf49-b22cd034a663 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:35.616642 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c935ce9-6567-491f-bf49-b22cd034a663 admin admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:35.617334 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c935ce9-6567-491f-bf49-b22cd034a663 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:35.619234 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c935ce9-6567-491f-bf49-b22cd034a663 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:35.642280 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1354/2733] 10.222.0.89 () {60 vars in 1144 bytes} [Thu Jul 28 17:54:35 2022] POST /identity/v3/users => generated 296 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:54:35.646361 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b514ac0a-9a94-49e0-80fb-cfc674a2d98e admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:35.657227 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b514ac0a-9a94-49e0-80fb-cfc674a2d98e admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:35.662171 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b402a2d-8866-4104-9ab5-8b0afcf3e1de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:35.662788 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b402a2d-8866-4104-9ab5-8b0afcf3e1de admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:35.663067 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b402a2d-8866-4104-9ab5-8b0afcf3e1de admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:35.663334 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b402a2d-8866-4104-9ab5-8b0afcf3e1de admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:35.664028 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b402a2d-8866-4104-9ab5-8b0afcf3e1de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:35.665741 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b402a2d-8866-4104-9ab5-8b0afcf3e1de admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:35.678437 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1380/2734] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:54:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:54:35.682291 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c935ce9-6567-491f-bf49-b22cd034a663 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:35.693947 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c935ce9-6567-491f-bf49-b22cd034a663 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:35.699293 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4fec5e-d002-48b6-8205-4263332f4fb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:35.700177 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4fec5e-d002-48b6-8205-4263332f4fb9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:35.700624 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4fec5e-d002-48b6-8205-4263332f4fb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:35.700990 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4fec5e-d002-48b6-8205-4263332f4fb9 admin admin] PATH_INFO: `/v3/projects/a36fe479443540dfbecf9373a72ca18d/users/07d6b2e80ab1411594bacded08511ed4/roles/73e457231a824ea98c28d22610be640c` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:35.725067 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4fec5e-d002-48b6-8205-4263332f4fb9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a36fe479443540dfbecf9373a72ca18d, user_id=07d6b2e80ab1411594bacded08511ed4, role_id=73e457231a824ea98c28d22610be640c)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:35.728323 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4fec5e-d002-48b6-8205-4263332f4fb9 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:35.750406 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1355/2735] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:54:35 2022] PUT /identity/v3/projects/a36fe479443540dfbecf9373a72ca18d/users/07d6b2e80ab1411594bacded08511ed4/roles/73e457231a824ea98c28d22610be640c => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 17:54:35.754473 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b402a2d-8866-4104-9ab5-8b0afcf3e1de admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:35.764614 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b402a2d-8866-4104-9ab5-8b0afcf3e1de admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:35.833189 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdf0fbb-2655-4a38-87c2-8d0afcfc1438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:35.833829 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdf0fbb-2655-4a38-87c2-8d0afcfc1438 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:35.834107 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdf0fbb-2655-4a38-87c2-8d0afcfc1438 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:35.834376 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdf0fbb-2655-4a38-87c2-8d0afcfc1438 admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:35.835067 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdf0fbb-2655-4a38-87c2-8d0afcfc1438 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:35.836687 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdf0fbb-2655-4a38-87c2-8d0afcfc1438 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:35.850483 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1381/2736] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:54:35 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:54:35.854834 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4fec5e-d002-48b6-8205-4263332f4fb9 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:35.866237 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4fec5e-d002-48b6-8205-4263332f4fb9 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:35.871002 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92441f9-3e62-408e-b4b3-b02beab28ef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:35.871671 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92441f9-3e62-408e-b4b3-b02beab28ef2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:35.871950 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92441f9-3e62-408e-b4b3-b02beab28ef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:35.872223 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92441f9-3e62-408e-b4b3-b02beab28ef2 admin admin] PATH_INFO: `/v3/projects/a36fe479443540dfbecf9373a72ca18d/users/07d6b2e80ab1411594bacded08511ed4/roles/73e457231a824ea98c28d22610be640c` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:35.874344 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92441f9-3e62-408e-b4b3-b02beab28ef2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a36fe479443540dfbecf9373a72ca18d, user_id=07d6b2e80ab1411594bacded08511ed4, role_id=73e457231a824ea98c28d22610be640c)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:35.877130 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92441f9-3e62-408e-b4b3-b02beab28ef2 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:35.888676 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1356/2737] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:54:35 2022] PUT /identity/v3/projects/a36fe479443540dfbecf9373a72ca18d/users/07d6b2e80ab1411594bacded08511ed4/roles/73e457231a824ea98c28d22610be640c => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:54:40.889749 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdf0fbb-2655-4a38-87c2-8d0afcfc1438 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:40.893081 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-9cdf0fbb-2655-4a38-87c2-8d0afcfc1438 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:54:40.898176 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a17d32-7038-4b7f-b404-bd734eb5aa82 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:40.898991 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a17d32-7038-4b7f-b404-bd734eb5aa82 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:40.899459 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a17d32-7038-4b7f-b404-bd734eb5aa82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:40.929881 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-05a17d32-7038-4b7f-b404-bd734eb5aa82 None None] MFA Rules not processed for user `07d6b2e80ab1411594bacded08511ed4`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:54:40.932047 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-05a17d32-7038-4b7f-b404-bd734eb5aa82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:54:40.964627 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdf0fbb-2655-4a38-87c2-8d0afcfc1438 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:41.000902 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:41.003745 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:41.017094 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1357/2738] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:54:40 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 17:54:41.030927 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a17d32-7038-4b7f-b404-bd734eb5aa82 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:41.032549 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-05a17d32-7038-4b7f-b404-bd734eb5aa82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:54:41.039023 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd765d4-cf13-4a1e-9aa3-c6d8c93b5a88 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96d512d0c84240918dae6c4a3e8b7b79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:41.039643 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd765d4-cf13-4a1e-9aa3-c6d8c93b5a88 service cinder] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:41.039922 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd765d4-cf13-4a1e-9aa3-c6d8c93b5a88 service cinder] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:41.040193 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd765d4-cf13-4a1e-9aa3-c6d8c93b5a88 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:41.042543 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-4dd765d4-cf13-4a1e-9aa3-c6d8c93b5a88 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:54:41.118158 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a17d32-7038-4b7f-b404-bd734eb5aa82 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:41.124208 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7a5bbc-776f-4bc7-9220-7071481ea909 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd10b49b29df94404a71f43273839c252', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:41.124978 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7a5bbc-776f-4bc7-9220-7071481ea909 service nova] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:41.125713 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7a5bbc-776f-4bc7-9220-7071481ea909 service nova] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:41.126057 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7a5bbc-776f-4bc7-9220-7071481ea909 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:41.127109 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd765d4-cf13-4a1e-9aa3-c6d8c93b5a88 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:41.128568 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd765d4-cf13-4a1e-9aa3-c6d8c93b5a88 service cinder] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:41.142304 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1382/2739] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:54:40 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:54:41.205900 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7a5bbc-776f-4bc7-9220-7071481ea909 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:41.207681 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7a5bbc-776f-4bc7-9220-7071481ea909 service nova] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:41.217518 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1358/2740] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:54:41 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 17:54:49.061413 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd765d4-cf13-4a1e-9aa3-c6d8c93b5a88 service cinder] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:54:49.062825 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-4dd765d4-cf13-4a1e-9aa3-c6d8c93b5a88 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:54:49.074701 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd765d4-cf13-4a1e-9aa3-c6d8c93b5a88 service cinder] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:54:49.078366 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6b0665-a122-4d86-9494-f54a5d0a1368 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96d512d0c84240918dae6c4a3e8b7b79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:54:49.078975 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6b0665-a122-4d86-9494-f54a5d0a1368 service cinder] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:49.079256 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6b0665-a122-4d86-9494-f54a5d0a1368 service cinder] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:49.079529 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6b0665-a122-4d86-9494-f54a5d0a1368 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:49.082118 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-6d6b0665-a122-4d86-9494-f54a5d0a1368 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:54:49.097635 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6b0665-a122-4d86-9494-f54a5d0a1368 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:54:49.099083 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6b0665-a122-4d86-9494-f54a5d0a1368 service cinder] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:54:49.110712 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1383/2741] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:54:49 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:54:51.212831 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=58 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:54:57.698226 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b9e04a-6d7b-44ef-b98c-ccddeef06812 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:57.698787 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b9e04a-6d7b-44ef-b98c-ccddeef06812 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:57.698904 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b9e04a-6d7b-44ef-b98c-ccddeef06812 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:57.720064 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-a5b9e04a-6d7b-44ef-b98c-ccddeef06812 None None] MFA Rules not processed for user `39ff02cba2fd4e7b9fb27b98b9daa82e`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:54:57.722226 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-a5b9e04a-6d7b-44ef-b98c-ccddeef06812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:54:57.787239 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1359/2742] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:54:57 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 17:54:58.847057 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad1213a-bfce-47bd-9107-09a76a4c2f29 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:54:58.847629 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad1213a-bfce-47bd-9107-09a76a4c2f29 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:54:58.847629 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad1213a-bfce-47bd-9107-09a76a4c2f29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:54:58.868647 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-2ad1213a-bfce-47bd-9107-09a76a4c2f29 None None] MFA Rules not processed for user `c4618ca07ce2408ba90e9eaab0b940c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:54:58.869920 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-2ad1213a-bfce-47bd-9107-09a76a4c2f29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:54:58.878018 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1384/2743] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:54:58 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:55:07.778773 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=25 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:55:08.862374 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=48 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:55:16.693311 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b9e04a-6d7b-44ef-b98c-ccddeef06812 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:55:16.694763 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-a5b9e04a-6d7b-44ef-b98c-ccddeef06812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:55:16.750979 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b9e04a-6d7b-44ef-b98c-ccddeef06812 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:55:16.754572 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8769f755-caa4-4b50-a580-3c66c2ba3647 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76817cd541294331b0023069092e8e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:55:16.755183 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8769f755-caa4-4b50-a580-3c66c2ba3647 service neutron] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:55:16.755461 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8769f755-caa4-4b50-a580-3c66c2ba3647 service neutron] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:55:16.755733 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8769f755-caa4-4b50-a580-3c66c2ba3647 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:55:16.764592 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8769f755-caa4-4b50-a580-3c66c2ba3647 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:55:16.766035 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8769f755-caa4-4b50-a580-3c66c2ba3647 service neutron] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:55:16.775677 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1360/2744] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:55:16 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:55:18.729261 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad1213a-bfce-47bd-9107-09a76a4c2f29 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:55:18.741906 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad1213a-bfce-47bd-9107-09a76a4c2f29 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:55:18.745855 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89659aa7-c142-4f13-acb1-d9b4191ceae9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76817cd541294331b0023069092e8e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:55:18.746484 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89659aa7-c142-4f13-acb1-d9b4191ceae9 service neutron] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:55:18.746758 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89659aa7-c142-4f13-acb1-d9b4191ceae9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:55:18.747027 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89659aa7-c142-4f13-acb1-d9b4191ceae9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:55:18.748869 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-89659aa7-c142-4f13-acb1-d9b4191ceae9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:55:18.761016 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89659aa7-c142-4f13-acb1-d9b4191ceae9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:55:18.762564 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89659aa7-c142-4f13-acb1-d9b4191ceae9 service neutron] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:55:18.772570 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1385/2745] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:55:18 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 17:55:26.771089 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=26 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:55:28.767715 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=6 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:55:52.940811 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8769f755-caa4-4b50-a580-3c66c2ba3647 service neutron] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:55:52.953306 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8769f755-caa4-4b50-a580-3c66c2ba3647 service neutron] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:55:52.958611 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014946d2-a655-420b-8dbc-340d708031cd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd10b49b29df94404a71f43273839c252', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:55:52.959240 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014946d2-a655-420b-8dbc-340d708031cd service nova] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:55:52.959525 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014946d2-a655-420b-8dbc-340d708031cd service nova] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:55:52.960329 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014946d2-a655-420b-8dbc-340d708031cd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:55:52.962398 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-014946d2-a655-420b-8dbc-340d708031cd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:55:52.973245 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014946d2-a655-420b-8dbc-340d708031cd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:55:52.974958 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014946d2-a655-420b-8dbc-340d708031cd service nova] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:55:52.984648 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1361/2746] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:55:52 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:56:02.980470 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=6 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:56:07.023433 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89659aa7-c142-4f13-acb1-d9b4191ceae9 service neutron] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:07.025688 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-89659aa7-c142-4f13-acb1-d9b4191ceae9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:56:07.043628 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89659aa7-c142-4f13-acb1-d9b4191ceae9 service neutron] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:07.050170 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b84b61-325e-49c1-9ac1-d514bc239e73 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd10b49b29df94404a71f43273839c252', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:07.050802 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b84b61-325e-49c1-9ac1-d514bc239e73 service nova] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:07.051090 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b84b61-325e-49c1-9ac1-d514bc239e73 service nova] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:07.051346 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b84b61-325e-49c1-9ac1-d514bc239e73 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:07.054313 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-16b84b61-325e-49c1-9ac1-d514bc239e73 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:56:07.068101 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b84b61-325e-49c1-9ac1-d514bc239e73 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:07.070605 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b84b61-325e-49c1-9ac1-d514bc239e73 service nova] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:07.082542 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1386/2747] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:56:07 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:56:12.749956 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014946d2-a655-420b-8dbc-340d708031cd service nova] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:12.759797 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014946d2-a655-420b-8dbc-340d708031cd service nova] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:12.763361 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5845b04a-7431-4b4a-9e41-70b9a943c636 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76817cd541294331b0023069092e8e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:12.763974 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5845b04a-7431-4b4a-9e41-70b9a943c636 service neutron] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:12.764254 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5845b04a-7431-4b4a-9e41-70b9a943c636 service neutron] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:12.764527 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5845b04a-7431-4b4a-9e41-70b9a943c636 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:12.766309 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-5845b04a-7431-4b4a-9e41-70b9a943c636 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:56:12.831935 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5845b04a-7431-4b4a-9e41-70b9a943c636 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:12.833390 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5845b04a-7431-4b4a-9e41-70b9a943c636 service neutron] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:12.843167 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1362/2748] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:56:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:56:16.105121 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b84b61-325e-49c1-9ac1-d514bc239e73 service nova] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:16.106763 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-16b84b61-325e-49c1-9ac1-d514bc239e73 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:56:16.123138 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b84b61-325e-49c1-9ac1-d514bc239e73 service nova] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:16.127707 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad68d35-d34f-4d10-b7b5-3f9c53136be4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:16.128337 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad68d35-d34f-4d10-b7b5-3f9c53136be4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:16.128617 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad68d35-d34f-4d10-b7b5-3f9c53136be4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:16.128921 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad68d35-d34f-4d10-b7b5-3f9c53136be4 admin admin] PATH_INFO: `/v3/users/39ff02cba2fd4e7b9fb27b98b9daa82e` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:16.130138 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad68d35-d34f-4d10-b7b5-3f9c53136be4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=39ff02cba2fd4e7b9fb27b98b9daa82e)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:16.132296 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad68d35-d34f-4d10-b7b5-3f9c53136be4 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:16.185436 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-6ad68d35-d34f-4d10-b7b5-3f9c53136be4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '39ff02cba2fd4e7b9fb27b98b9daa82e'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:56:16.192286 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-6ad68d35-d34f-4d10-b7b5-3f9c53136be4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '39ff02cba2fd4e7b9fb27b98b9daa82e'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:56:16.192960 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-6ad68d35-d34f-4d10-b7b5-3f9c53136be4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '39ff02cba2fd4e7b9fb27b98b9daa82e'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:56:16.200992 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-6ad68d35-d34f-4d10-b7b5-3f9c53136be4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '39ff02cba2fd4e7b9fb27b98b9daa82e'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:56:16.218560 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-6ad68d35-d34f-4d10-b7b5-3f9c53136be4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '39ff02cba2fd4e7b9fb27b98b9daa82e'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:56:16.226321 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1387/2749] 10.222.0.89 () {58 vars in 1224 bytes} [Thu Jul 28 17:56:16 2022] DELETE /identity/v3/users/39ff02cba2fd4e7b9fb27b98b9daa82e => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:56:16.408400 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5845b04a-7431-4b4a-9e41-70b9a943c636 service neutron] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:16.420700 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5845b04a-7431-4b4a-9e41-70b9a943c636 service neutron] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:16.479427 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83068345-76f7-4a59-a61c-3030e917d38f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:16.480058 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83068345-76f7-4a59-a61c-3030e917d38f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:16.480332 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83068345-76f7-4a59-a61c-3030e917d38f admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:16.480604 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83068345-76f7-4a59-a61c-3030e917d38f admin admin] PATH_INFO: `/v3/projects/72a9ad9df4f84c658e9964f1a0c0b282` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:16.481904 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83068345-76f7-4a59-a61c-3030e917d38f admin admin] RBAC: Authorizing `identity:delete_project(project_id=72a9ad9df4f84c658e9964f1a0c0b282)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:16.483820 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83068345-76f7-4a59-a61c-3030e917d38f admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:16.530444 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-83068345-76f7-4a59-a61c-3030e917d38f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72a9ad9df4f84c658e9964f1a0c0b282', 'request_id': 'req-83068345-76f7-4a59-a61c-3030e917d38f'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:56:16.536511 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-83068345-76f7-4a59-a61c-3030e917d38f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72a9ad9df4f84c658e9964f1a0c0b282', 'request_id': 'req-83068345-76f7-4a59-a61c-3030e917d38f'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:56:16.542314 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-83068345-76f7-4a59-a61c-3030e917d38f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72a9ad9df4f84c658e9964f1a0c0b282', 'request_id': 'req-83068345-76f7-4a59-a61c-3030e917d38f'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:56:16.554884 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-83068345-76f7-4a59-a61c-3030e917d38f admin admin] The token cache is being invalidate because project 72a9ad9df4f84c658e9964f1a0c0b282 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80325) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 17:56:16.555172 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-83068345-76f7-4a59-a61c-3030e917d38f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:56:16.555798 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-83068345-76f7-4a59-a61c-3030e917d38f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:56:16.557349 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1363/2750] 10.222.0.89 () {58 vars in 1233 bytes} [Thu Jul 28 17:56:16 2022] DELETE /identity/v3/projects/72a9ad9df4f84c658e9964f1a0c0b282 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:56:16.562237 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cf3e07-b6a6-4996-a5e3-4bf55a4d1cee None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:16.562527 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cf3e07-b6a6-4996-a5e3-4bf55a4d1cee None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:16.562809 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cf3e07-b6a6-4996-a5e3-4bf55a4d1cee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:16.584838 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-18cf3e07-b6a6-4996-a5e3-4bf55a4d1cee None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:56:16.586113 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-18cf3e07-b6a6-4996-a5e3-4bf55a4d1cee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:56:16.647865 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1388/2751] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:56:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:56:16.655985 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8b7bdf-7cc5-4aa5-8a6d-ba7762f8101a None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:16.656268 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8b7bdf-7cc5-4aa5-8a6d-ba7762f8101a None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:16.656566 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8b7bdf-7cc5-4aa5-8a6d-ba7762f8101a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:16.676025 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-4b8b7bdf-7cc5-4aa5-8a6d-ba7762f8101a None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:56:16.677257 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-4b8b7bdf-7cc5-4aa5-8a6d-ba7762f8101a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:56:16.685627 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1364/2752] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:56:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:56:16.689851 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cf3e07-b6a6-4996-a5e3-4bf55a4d1cee None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:16.700329 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cf3e07-b6a6-4996-a5e3-4bf55a4d1cee None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:16.704999 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1503ef0c-fbfa-4f63-8ec1-57f469c4d1ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:16.705634 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1503ef0c-fbfa-4f63-8ec1-57f469c4d1ff admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:16.705944 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1503ef0c-fbfa-4f63-8ec1-57f469c4d1ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:16.706218 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1503ef0c-fbfa-4f63-8ec1-57f469c4d1ff admin admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:16.706916 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1503ef0c-fbfa-4f63-8ec1-57f469c4d1ff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:56:16.707223 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1503ef0c-fbfa-4f63-8ec1-57f469c4d1ff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:16.708844 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1503ef0c-fbfa-4f63-8ec1-57f469c4d1ff admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:16.720889 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1389/2753] 10.222.0.89 () {58 vars in 1153 bytes} [Thu Jul 28 17:56:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:56:16.725707 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8b7bdf-7cc5-4aa5-8a6d-ba7762f8101a None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:16.734873 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8b7bdf-7cc5-4aa5-8a6d-ba7762f8101a None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:16.739519 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fa0412-a96f-4835-a8be-4832399258c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:16.740170 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fa0412-a96f-4835-a8be-4832399258c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:16.740548 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fa0412-a96f-4835-a8be-4832399258c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:16.741023 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fa0412-a96f-4835-a8be-4832399258c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:16.741727 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fa0412-a96f-4835-a8be-4832399258c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:16.743604 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fa0412-a96f-4835-a8be-4832399258c8 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:16.759661 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:56:16.774382 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1365/2754] 10.222.0.89 () {60 vars in 1153 bytes} [Thu Jul 28 17:56:16 2022] POST /identity/v3/projects => generated 405 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:56:16.778838 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1503ef0c-fbfa-4f63-8ec1-57f469c4d1ff admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:16.788839 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1503ef0c-fbfa-4f63-8ec1-57f469c4d1ff admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:16.833659 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:56:16.848285 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94636aa-b3c9-4a0d-a4ac-ba1a425c595d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:16.848918 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94636aa-b3c9-4a0d-a4ac-ba1a425c595d admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:16.849215 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94636aa-b3c9-4a0d-a4ac-ba1a425c595d admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:16.849488 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94636aa-b3c9-4a0d-a4ac-ba1a425c595d admin admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:16.850380 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94636aa-b3c9-4a0d-a4ac-ba1a425c595d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:16.852275 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94636aa-b3c9-4a0d-a4ac-ba1a425c595d admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:16.874814 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1390/2755] 10.222.0.89 () {60 vars in 1144 bytes} [Thu Jul 28 17:56:16 2022] POST /identity/v3/users => generated 303 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:56:16.878896 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fa0412-a96f-4835-a8be-4832399258c8 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:16.890089 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fa0412-a96f-4835-a8be-4832399258c8 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:16.894815 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ece7b9d-74c7-4225-9054-e1a95e540341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:16.895419 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ece7b9d-74c7-4225-9054-e1a95e540341 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:16.895696 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ece7b9d-74c7-4225-9054-e1a95e540341 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:16.895971 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ece7b9d-74c7-4225-9054-e1a95e540341 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:16.897268 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ece7b9d-74c7-4225-9054-e1a95e540341 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:16.898897 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ece7b9d-74c7-4225-9054-e1a95e540341 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:16.910067 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1366/2756] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:56:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:56:16.914409 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94636aa-b3c9-4a0d-a4ac-ba1a425c595d admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:16.924727 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94636aa-b3c9-4a0d-a4ac-ba1a425c595d admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:16.930395 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac85fba5-5e80-434c-93ff-af9806934dde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:16.931004 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac85fba5-5e80-434c-93ff-af9806934dde admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:16.931286 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac85fba5-5e80-434c-93ff-af9806934dde admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:16.931564 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac85fba5-5e80-434c-93ff-af9806934dde admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:16.932265 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac85fba5-5e80-434c-93ff-af9806934dde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:16.933904 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac85fba5-5e80-434c-93ff-af9806934dde admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:16.945201 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1391/2757] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:56:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:56:16.949392 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ece7b9d-74c7-4225-9054-e1a95e540341 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:16.959489 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ece7b9d-74c7-4225-9054-e1a95e540341 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:16.964331 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dceba2f7-e4cd-4c82-b4a4-c93397066aaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:16.964991 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceba2f7-e4cd-4c82-b4a4-c93397066aaf admin admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:16.965309 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceba2f7-e4cd-4c82-b4a4-c93397066aaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:16.965590 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceba2f7-e4cd-4c82-b4a4-c93397066aaf admin admin] PATH_INFO: `/v3/projects/3540fe6de7af44b18b327294143516e2/users/bd9167a02b3545a1a277d35359da7220/roles/11f8731756fe4caca13efa61b71b11c5` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:16.980355 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dceba2f7-e4cd-4c82-b4a4-c93397066aaf admin admin] RBAC: Authorizing `identity:create_grant(project_id=3540fe6de7af44b18b327294143516e2, user_id=bd9167a02b3545a1a277d35359da7220, role_id=11f8731756fe4caca13efa61b71b11c5)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:16.983055 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dceba2f7-e4cd-4c82-b4a4-c93397066aaf admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:16.997218 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1367/2758] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:56:16 2022] PUT /identity/v3/projects/3540fe6de7af44b18b327294143516e2/users/bd9167a02b3545a1a277d35359da7220/roles/11f8731756fe4caca13efa61b71b11c5 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:56:17.008561 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dc7454-760b-45c2-810f-3b8ee191df66 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:17.008848 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dc7454-760b-45c2-810f-3b8ee191df66 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:17.009120 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dc7454-760b-45c2-810f-3b8ee191df66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:17.036749 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-e6dc7454-760b-45c2-810f-3b8ee191df66 None None] MFA Rules not processed for user `bd9167a02b3545a1a277d35359da7220`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:56:17.038190 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-e6dc7454-760b-45c2-810f-3b8ee191df66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:56:17.102095 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1392/2759] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:56:17 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 17:56:17.106542 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dceba2f7-e4cd-4c82-b4a4-c93397066aaf admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:17.169757 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dceba2f7-e4cd-4c82-b4a4-c93397066aaf admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:17.174465 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d243734-fbf7-4dc6-8edd-522f25128f74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:17.175092 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d243734-fbf7-4dc6-8edd-522f25128f74 admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:17.175373 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d243734-fbf7-4dc6-8edd-522f25128f74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:17.175647 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d243734-fbf7-4dc6-8edd-522f25128f74 admin admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:17.176330 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d243734-fbf7-4dc6-8edd-522f25128f74 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:17.178987 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d243734-fbf7-4dc6-8edd-522f25128f74 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:17.205761 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1368/2760] 10.222.0.89 () {60 vars in 1153 bytes} [Thu Jul 28 17:56:17 2022] POST /identity/v3/projects => generated 403 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:56:17.210311 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dc7454-760b-45c2-810f-3b8ee191df66 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:17.221493 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dc7454-760b-45c2-810f-3b8ee191df66 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:17.278499 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b9bb30-21fc-473d-91f5-1b4e5513f05c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:17.279109 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b9bb30-21fc-473d-91f5-1b4e5513f05c admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:17.279389 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b9bb30-21fc-473d-91f5-1b4e5513f05c admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:17.279662 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b9bb30-21fc-473d-91f5-1b4e5513f05c admin admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:17.280341 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b9bb30-21fc-473d-91f5-1b4e5513f05c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:17.282265 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b9bb30-21fc-473d-91f5-1b4e5513f05c admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:17.305434 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1393/2761] 10.222.0.89 () {60 vars in 1144 bytes} [Thu Jul 28 17:56:17 2022] POST /identity/v3/users => generated 308 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 17:56:17.309618 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d243734-fbf7-4dc6-8edd-522f25128f74 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:17.320762 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d243734-fbf7-4dc6-8edd-522f25128f74 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:17.325401 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b7f4f0-a1fc-46ae-a5e8-f9e3fd8e0798 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:17.326003 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b7f4f0-a1fc-46ae-a5e8-f9e3fd8e0798 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:17.326284 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b7f4f0-a1fc-46ae-a5e8-f9e3fd8e0798 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:17.326557 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b7f4f0-a1fc-46ae-a5e8-f9e3fd8e0798 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:17.327248 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b7f4f0-a1fc-46ae-a5e8-f9e3fd8e0798 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:17.328825 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b7f4f0-a1fc-46ae-a5e8-f9e3fd8e0798 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:17.340116 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1369/2762] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:56:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:56:17.344315 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b9bb30-21fc-473d-91f5-1b4e5513f05c admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:17.355066 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b9bb30-21fc-473d-91f5-1b4e5513f05c admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:17.359772 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45687a8-f840-4b68-bc70-a97212f7988a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:17.360427 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45687a8-f840-4b68-bc70-a97212f7988a admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:17.360709 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45687a8-f840-4b68-bc70-a97212f7988a admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:17.360984 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45687a8-f840-4b68-bc70-a97212f7988a admin admin] PATH_INFO: `/v3/projects/8fe8f22b784e46bdad3b7a172ffa2d3d/users/2ba0ecb949a447e0b1c1f5553629dacf/roles/73e457231a824ea98c28d22610be640c` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:17.379299 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45687a8-f840-4b68-bc70-a97212f7988a admin admin] RBAC: Authorizing `identity:create_grant(project_id=8fe8f22b784e46bdad3b7a172ffa2d3d, user_id=2ba0ecb949a447e0b1c1f5553629dacf, role_id=73e457231a824ea98c28d22610be640c)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:17.382262 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45687a8-f840-4b68-bc70-a97212f7988a admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:17.395362 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1394/2763] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:56:17 2022] PUT /identity/v3/projects/8fe8f22b784e46bdad3b7a172ffa2d3d/users/2ba0ecb949a447e0b1c1f5553629dacf/roles/73e457231a824ea98c28d22610be640c => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:56:17.399674 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b7f4f0-a1fc-46ae-a5e8-f9e3fd8e0798 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:17.410460 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b7f4f0-a1fc-46ae-a5e8-f9e3fd8e0798 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:17.474411 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f2d6cf-b2be-49f7-ada0-9f10ac8acac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:17.475802 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f2d6cf-b2be-49f7-ada0-9f10ac8acac3 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:17.476102 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f2d6cf-b2be-49f7-ada0-9f10ac8acac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:17.476378 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f2d6cf-b2be-49f7-ada0-9f10ac8acac3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:17.477097 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f2d6cf-b2be-49f7-ada0-9f10ac8acac3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:17.478749 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f2d6cf-b2be-49f7-ada0-9f10ac8acac3 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:17.488539 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1370/2764] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:56:17 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:56:17.493469 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45687a8-f840-4b68-bc70-a97212f7988a admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:17.503955 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45687a8-f840-4b68-bc70-a97212f7988a admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:17.508860 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984bfad4-6aa3-4e8e-a111-dd1bc572d5f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:17.509553 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984bfad4-6aa3-4e8e-a111-dd1bc572d5f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:17.509830 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984bfad4-6aa3-4e8e-a111-dd1bc572d5f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:17.510102 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984bfad4-6aa3-4e8e-a111-dd1bc572d5f2 admin admin] PATH_INFO: `/v3/projects/8fe8f22b784e46bdad3b7a172ffa2d3d/users/2ba0ecb949a447e0b1c1f5553629dacf/roles/73e457231a824ea98c28d22610be640c` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:17.512055 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984bfad4-6aa3-4e8e-a111-dd1bc572d5f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8fe8f22b784e46bdad3b7a172ffa2d3d, user_id=2ba0ecb949a447e0b1c1f5553629dacf, role_id=73e457231a824ea98c28d22610be640c)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:17.514726 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984bfad4-6aa3-4e8e-a111-dd1bc572d5f2 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:17.524963 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1395/2765] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:56:17 2022] PUT /identity/v3/projects/8fe8f22b784e46bdad3b7a172ffa2d3d/users/2ba0ecb949a447e0b1c1f5553629dacf/roles/73e457231a824ea98c28d22610be640c => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:56:17.591483 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5f8cdf-b51d-41b3-b14b-ab2945bdd21b None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:17.591798 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5f8cdf-b51d-41b3-b14b-ab2945bdd21b None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:17.592079 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5f8cdf-b51d-41b3-b14b-ab2945bdd21b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:17.620574 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-dd5f8cdf-b51d-41b3-b14b-ab2945bdd21b None None] MFA Rules not processed for user `2ba0ecb949a447e0b1c1f5553629dacf`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:56:17.621835 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-dd5f8cdf-b51d-41b3-b14b-ab2945bdd21b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:56:17.640146 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:56:17.691627 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1371/2766] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:56:17 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:56:17.702399 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984bfad4-6aa3-4e8e-a111-dd1bc572d5f2 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:17.704255 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-984bfad4-6aa3-4e8e-a111-dd1bc572d5f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:56:17.714820 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:56:17.764760 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984bfad4-6aa3-4e8e-a111-dd1bc572d5f2 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:17.768361 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c18748f-472c-49a7-8496-c5973dcf9da4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96d512d0c84240918dae6c4a3e8b7b79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:17.768969 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c18748f-472c-49a7-8496-c5973dcf9da4 service cinder] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:17.769280 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c18748f-472c-49a7-8496-c5973dcf9da4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:17.769557 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c18748f-472c-49a7-8496-c5973dcf9da4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:17.780511 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c18748f-472c-49a7-8496-c5973dcf9da4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:17.782125 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c18748f-472c-49a7-8496-c5973dcf9da4 service cinder] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:17.792372 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1396/2767] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:56:17 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:56:17.942562 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5f8cdf-b51d-41b3-b14b-ab2945bdd21b None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:18.008873 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5f8cdf-b51d-41b3-b14b-ab2945bdd21b None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:18.013418 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7918a934-a2db-4141-a8d0-6e1a3bc06a82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:18.014063 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7918a934-a2db-4141-a8d0-6e1a3bc06a82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:18.014337 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7918a934-a2db-4141-a8d0-6e1a3bc06a82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:18.014609 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7918a934-a2db-4141-a8d0-6e1a3bc06a82 admin admin] PATH_INFO: `/v3/users/bd9167a02b3545a1a277d35359da7220` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:18.015963 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7918a934-a2db-4141-a8d0-6e1a3bc06a82 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd9167a02b3545a1a277d35359da7220)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:18.017735 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7918a934-a2db-4141-a8d0-6e1a3bc06a82 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:18.064130 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-7918a934-a2db-4141-a8d0-6e1a3bc06a82 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd9167a02b3545a1a277d35359da7220'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:56:18.071486 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-7918a934-a2db-4141-a8d0-6e1a3bc06a82 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd9167a02b3545a1a277d35359da7220'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:56:18.079018 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-7918a934-a2db-4141-a8d0-6e1a3bc06a82 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd9167a02b3545a1a277d35359da7220'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:56:18.079670 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-7918a934-a2db-4141-a8d0-6e1a3bc06a82 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd9167a02b3545a1a277d35359da7220'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:56:18.080278 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-7918a934-a2db-4141-a8d0-6e1a3bc06a82 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd9167a02b3545a1a277d35359da7220'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:56:18.098017 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1372/2768] 10.222.0.89 () {58 vars in 1224 bytes} [Thu Jul 28 17:56:17 2022] DELETE /identity/v3/users/bd9167a02b3545a1a277d35359da7220 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:56:18.102817 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e2486d-195b-4c6f-9965-811e4b97cc4c None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:18.103101 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e2486d-195b-4c6f-9965-811e4b97cc4c None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:18.103378 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e2486d-195b-4c6f-9965-811e4b97cc4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:18.122716 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-99e2486d-195b-4c6f-9965-811e4b97cc4c None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:56:18.123909 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-99e2486d-195b-4c6f-9965-811e4b97cc4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:56:18.182223 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1397/2769] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:56:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:56:18.191836 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7918a934-a2db-4141-a8d0-6e1a3bc06a82 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:18.193797 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-7918a934-a2db-4141-a8d0-6e1a3bc06a82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:56:18.235411 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:56:18.254724 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7918a934-a2db-4141-a8d0-6e1a3bc06a82 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:18.259267 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1934e2-69d6-49a3-a03a-481a87e650bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76817cd541294331b0023069092e8e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:18.259878 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1934e2-69d6-49a3-a03a-481a87e650bb service neutron] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:18.260172 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1934e2-69d6-49a3-a03a-481a87e650bb service neutron] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:18.260430 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1934e2-69d6-49a3-a03a-481a87e650bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:18.269739 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1934e2-69d6-49a3-a03a-481a87e650bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:18.271091 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1934e2-69d6-49a3-a03a-481a87e650bb service neutron] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:18.281748 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1373/2770] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:56:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:56:18.557868 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e2486d-195b-4c6f-9965-811e4b97cc4c None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:18.559619 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-99e2486d-195b-4c6f-9965-811e4b97cc4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:56:18.572727 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e2486d-195b-4c6f-9965-811e4b97cc4c None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:18.577642 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4557efb7-fd28-4221-ae8f-bce4787a9b5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:18.578295 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4557efb7-fd28-4221-ae8f-bce4787a9b5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:18.578587 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4557efb7-fd28-4221-ae8f-bce4787a9b5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:18.578856 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4557efb7-fd28-4221-ae8f-bce4787a9b5b admin admin] PATH_INFO: `/v3/projects/3540fe6de7af44b18b327294143516e2` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:18.580050 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4557efb7-fd28-4221-ae8f-bce4787a9b5b admin admin] RBAC: Authorizing `identity:delete_project(project_id=3540fe6de7af44b18b327294143516e2)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:18.582129 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4557efb7-fd28-4221-ae8f-bce4787a9b5b admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:18.630432 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-4557efb7-fd28-4221-ae8f-bce4787a9b5b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3540fe6de7af44b18b327294143516e2', 'request_id': 'req-4557efb7-fd28-4221-ae8f-bce4787a9b5b'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:56:18.636695 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-4557efb7-fd28-4221-ae8f-bce4787a9b5b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3540fe6de7af44b18b327294143516e2', 'request_id': 'req-4557efb7-fd28-4221-ae8f-bce4787a9b5b'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:56:18.642934 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-4557efb7-fd28-4221-ae8f-bce4787a9b5b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3540fe6de7af44b18b327294143516e2', 'request_id': 'req-4557efb7-fd28-4221-ae8f-bce4787a9b5b'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:56:18.657496 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-4557efb7-fd28-4221-ae8f-bce4787a9b5b admin admin] The token cache is being invalidate because project 3540fe6de7af44b18b327294143516e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80324) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 17:56:18.657786 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-4557efb7-fd28-4221-ae8f-bce4787a9b5b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:56:18.658410 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-4557efb7-fd28-4221-ae8f-bce4787a9b5b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:56:18.659992 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1398/2771] 10.222.0.89 () {58 vars in 1233 bytes} [Thu Jul 28 17:56:18 2022] DELETE /identity/v3/projects/3540fe6de7af44b18b327294143516e2 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:56:18.664063 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1934e2-69d6-49a3-a03a-481a87e650bb service neutron] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:18.673787 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1934e2-69d6-49a3-a03a-481a87e650bb service neutron] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:18.730785 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79de6ae8-97f7-4778-9ac0-991693f96103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:18.731434 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79de6ae8-97f7-4778-9ac0-991693f96103 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:18.731714 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79de6ae8-97f7-4778-9ac0-991693f96103 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:18.731993 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79de6ae8-97f7-4778-9ac0-991693f96103 admin admin] PATH_INFO: `/v3/users/2ba0ecb949a447e0b1c1f5553629dacf` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:18.733185 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79de6ae8-97f7-4778-9ac0-991693f96103 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ba0ecb949a447e0b1c1f5553629dacf)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:18.735119 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79de6ae8-97f7-4778-9ac0-991693f96103 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:18.788093 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-79de6ae8-97f7-4778-9ac0-991693f96103 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ba0ecb949a447e0b1c1f5553629dacf'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:56:18.796734 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-79de6ae8-97f7-4778-9ac0-991693f96103 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ba0ecb949a447e0b1c1f5553629dacf'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:56:18.804769 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-79de6ae8-97f7-4778-9ac0-991693f96103 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ba0ecb949a447e0b1c1f5553629dacf'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:56:18.805374 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-79de6ae8-97f7-4778-9ac0-991693f96103 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ba0ecb949a447e0b1c1f5553629dacf'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:56:18.806002 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-79de6ae8-97f7-4778-9ac0-991693f96103 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ba0ecb949a447e0b1c1f5553629dacf'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:56:18.823922 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1374/2772] 10.222.0.89 () {58 vars in 1224 bytes} [Thu Jul 28 17:56:18 2022] DELETE /identity/v3/users/2ba0ecb949a447e0b1c1f5553629dacf => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:56:19.096664 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4557efb7-fd28-4221-ae8f-bce4787a9b5b admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:19.098329 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-4557efb7-fd28-4221-ae8f-bce4787a9b5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:56:19.119833 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4557efb7-fd28-4221-ae8f-bce4787a9b5b admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:19.153718 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:56:19.181812 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271bd712-dc4e-4076-a39d-e08bfd5b81ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:19.182458 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271bd712-dc4e-4076-a39d-e08bfd5b81ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:19.182743 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271bd712-dc4e-4076-a39d-e08bfd5b81ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:19.183018 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271bd712-dc4e-4076-a39d-e08bfd5b81ca admin admin] PATH_INFO: `/v3/projects/8fe8f22b784e46bdad3b7a172ffa2d3d` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:19.184265 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271bd712-dc4e-4076-a39d-e08bfd5b81ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=8fe8f22b784e46bdad3b7a172ffa2d3d)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:19.186258 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271bd712-dc4e-4076-a39d-e08bfd5b81ca admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:19.232630 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-271bd712-dc4e-4076-a39d-e08bfd5b81ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8fe8f22b784e46bdad3b7a172ffa2d3d', 'request_id': 'req-271bd712-dc4e-4076-a39d-e08bfd5b81ca'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:56:19.238161 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-271bd712-dc4e-4076-a39d-e08bfd5b81ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8fe8f22b784e46bdad3b7a172ffa2d3d', 'request_id': 'req-271bd712-dc4e-4076-a39d-e08bfd5b81ca'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:56:19.244500 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-271bd712-dc4e-4076-a39d-e08bfd5b81ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8fe8f22b784e46bdad3b7a172ffa2d3d', 'request_id': 'req-271bd712-dc4e-4076-a39d-e08bfd5b81ca'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:56:19.257442 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-271bd712-dc4e-4076-a39d-e08bfd5b81ca admin admin] The token cache is being invalidate because project 8fe8f22b784e46bdad3b7a172ffa2d3d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80324) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 17:56:19.257727 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-271bd712-dc4e-4076-a39d-e08bfd5b81ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:56:19.258351 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-271bd712-dc4e-4076-a39d-e08bfd5b81ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:56:19.259862 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1399/2773] 10.222.0.89 () {58 vars in 1233 bytes} [Thu Jul 28 17:56:19 2022] DELETE /identity/v3/projects/8fe8f22b784e46bdad3b7a172ffa2d3d => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:56:19.264969 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674a411b-cfdb-48f3-980b-85b9981136a1 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:19.265289 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674a411b-cfdb-48f3-980b-85b9981136a1 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:19.265574 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674a411b-cfdb-48f3-980b-85b9981136a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:19.285923 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-674a411b-cfdb-48f3-980b-85b9981136a1 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:56:19.287754 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-674a411b-cfdb-48f3-980b-85b9981136a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:56:19.337048 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:56:19.348250 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1375/2774] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:56:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:56:19.356391 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f806988-2690-42a2-b370-975929d1de03 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:19.356679 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f806988-2690-42a2-b370-975929d1de03 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:19.356956 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f806988-2690-42a2-b370-975929d1de03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:19.377256 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-6f806988-2690-42a2-b370-975929d1de03 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:56:19.378507 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-6f806988-2690-42a2-b370-975929d1de03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:56:19.386562 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1400/2775] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:56:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:56:19.390821 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674a411b-cfdb-48f3-980b-85b9981136a1 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:19.401447 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674a411b-cfdb-48f3-980b-85b9981136a1 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:19.406127 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570b4cbd-2e1e-4204-9ede-dffa599dab6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:19.406740 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570b4cbd-2e1e-4204-9ede-dffa599dab6e admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:19.407018 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570b4cbd-2e1e-4204-9ede-dffa599dab6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:19.407296 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570b4cbd-2e1e-4204-9ede-dffa599dab6e admin admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:19.407975 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570b4cbd-2e1e-4204-9ede-dffa599dab6e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:56:19.408279 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570b4cbd-2e1e-4204-9ede-dffa599dab6e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:19.409983 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570b4cbd-2e1e-4204-9ede-dffa599dab6e admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:19.422273 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1376/2776] 10.222.0.89 () {58 vars in 1153 bytes} [Thu Jul 28 17:56:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:56:19.426629 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f806988-2690-42a2-b370-975929d1de03 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:19.436525 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f806988-2690-42a2-b370-975929d1de03 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:19.441496 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f84ea63-010c-4305-ad28-df3e05f66edc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:19.442109 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f84ea63-010c-4305-ad28-df3e05f66edc admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:19.442390 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f84ea63-010c-4305-ad28-df3e05f66edc admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:19.442669 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f84ea63-010c-4305-ad28-df3e05f66edc admin admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:19.444047 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f84ea63-010c-4305-ad28-df3e05f66edc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:19.446470 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f84ea63-010c-4305-ad28-df3e05f66edc admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:19.476228 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1401/2777] 10.222.0.89 () {60 vars in 1153 bytes} [Thu Jul 28 17:56:19 2022] POST /identity/v3/projects => generated 387 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:56:19.480585 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570b4cbd-2e1e-4204-9ede-dffa599dab6e admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:19.490657 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570b4cbd-2e1e-4204-9ede-dffa599dab6e admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:19.546241 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680db136-2964-49dc-802a-7c50a0ca54c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:19.546853 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680db136-2964-49dc-802a-7c50a0ca54c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:19.547137 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680db136-2964-49dc-802a-7c50a0ca54c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:19.547411 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680db136-2964-49dc-802a-7c50a0ca54c9 admin admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:19.548092 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680db136-2964-49dc-802a-7c50a0ca54c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:19.550030 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680db136-2964-49dc-802a-7c50a0ca54c9 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:19.575483 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1377/2778] 10.222.0.89 () {60 vars in 1144 bytes} [Thu Jul 28 17:56:19 2022] POST /identity/v3/users => generated 294 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 17:56:19.579758 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f84ea63-010c-4305-ad28-df3e05f66edc admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:19.590921 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f84ea63-010c-4305-ad28-df3e05f66edc admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:19.595591 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4b4f17-ff47-49c5-bb4a-7c412b7dbd82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:19.596197 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4b4f17-ff47-49c5-bb4a-7c412b7dbd82 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:19.596482 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4b4f17-ff47-49c5-bb4a-7c412b7dbd82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:19.596754 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4b4f17-ff47-49c5-bb4a-7c412b7dbd82 admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:19.597485 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4b4f17-ff47-49c5-bb4a-7c412b7dbd82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:19.599122 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4b4f17-ff47-49c5-bb4a-7c412b7dbd82 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:19.608976 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1402/2779] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:56:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:56:19.613424 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680db136-2964-49dc-802a-7c50a0ca54c9 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:19.623991 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680db136-2964-49dc-802a-7c50a0ca54c9 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:19.628825 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37e69d0-1115-4006-bd93-fef97ee52a3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:19.629466 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37e69d0-1115-4006-bd93-fef97ee52a3f admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:19.629750 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37e69d0-1115-4006-bd93-fef97ee52a3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:19.630025 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37e69d0-1115-4006-bd93-fef97ee52a3f admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:19.631220 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37e69d0-1115-4006-bd93-fef97ee52a3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:19.632806 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37e69d0-1115-4006-bd93-fef97ee52a3f admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:19.643654 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1378/2780] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:56:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 17:56:19.649733 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4b4f17-ff47-49c5-bb4a-7c412b7dbd82 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:19.660859 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4b4f17-ff47-49c5-bb4a-7c412b7dbd82 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:19.665912 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b557089-8b4b-4219-8fa4-e3e38592730d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:19.666571 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b557089-8b4b-4219-8fa4-e3e38592730d admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:19.666851 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b557089-8b4b-4219-8fa4-e3e38592730d admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:19.667127 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b557089-8b4b-4219-8fa4-e3e38592730d admin admin] PATH_INFO: `/v3/projects/e55fb51bd40f4a10b03f33a34acf3bd9/users/07f083cf9fae4837afd2ab90fe2cadff/roles/11f8731756fe4caca13efa61b71b11c5` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:19.681862 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b557089-8b4b-4219-8fa4-e3e38592730d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e55fb51bd40f4a10b03f33a34acf3bd9, user_id=07f083cf9fae4837afd2ab90fe2cadff, role_id=11f8731756fe4caca13efa61b71b11c5)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:19.684518 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b557089-8b4b-4219-8fa4-e3e38592730d admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:19.697893 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1403/2781] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:56:19 2022] PUT /identity/v3/projects/e55fb51bd40f4a10b03f33a34acf3bd9/users/07f083cf9fae4837afd2ab90fe2cadff/roles/11f8731756fe4caca13efa61b71b11c5 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:56:19.708131 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236eed04-73ee-4380-aeec-b92aca0538c5 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:19.708631 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236eed04-73ee-4380-aeec-b92aca0538c5 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:19.708790 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236eed04-73ee-4380-aeec-b92aca0538c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:19.736776 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-236eed04-73ee-4380-aeec-b92aca0538c5 None None] MFA Rules not processed for user `07f083cf9fae4837afd2ab90fe2cadff`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:56:19.738034 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-236eed04-73ee-4380-aeec-b92aca0538c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:56:19.803492 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1379/2782] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:56:19 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:56:19.808235 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b557089-8b4b-4219-8fa4-e3e38592730d admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:19.873949 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b557089-8b4b-4219-8fa4-e3e38592730d admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:19.878837 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9f9ec7-f62c-4083-aa7b-5aff2b2d487c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:19.879452 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9f9ec7-f62c-4083-aa7b-5aff2b2d487c admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:19.879732 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9f9ec7-f62c-4083-aa7b-5aff2b2d487c admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:19.880009 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9f9ec7-f62c-4083-aa7b-5aff2b2d487c admin admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:19.880678 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9f9ec7-f62c-4083-aa7b-5aff2b2d487c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:19.883241 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9f9ec7-f62c-4083-aa7b-5aff2b2d487c admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:19.887996 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:56:19.915480 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1404/2783] 10.222.0.89 () {60 vars in 1153 bytes} [Thu Jul 28 17:56:19 2022] POST /identity/v3/projects => generated 385 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:56:19.919598 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236eed04-73ee-4380-aeec-b92aca0538c5 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:19.931210 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236eed04-73ee-4380-aeec-b92aca0538c5 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:19.982881 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:56:19.993310 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd195c13-2b07-4fbf-af9f-82b5ad560e0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:19.993932 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd195c13-2b07-4fbf-af9f-82b5ad560e0e admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:19.994216 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd195c13-2b07-4fbf-af9f-82b5ad560e0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:19.994491 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd195c13-2b07-4fbf-af9f-82b5ad560e0e admin admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:19.995168 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd195c13-2b07-4fbf-af9f-82b5ad560e0e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:19.997104 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd195c13-2b07-4fbf-af9f-82b5ad560e0e admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:20.018515 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1380/2784] 10.222.0.89 () {60 vars in 1144 bytes} [Thu Jul 28 17:56:19 2022] POST /identity/v3/users => generated 299 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:56:20.022646 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9f9ec7-f62c-4083-aa7b-5aff2b2d487c admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:20.033547 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9f9ec7-f62c-4083-aa7b-5aff2b2d487c admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:20.038332 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23da3568-52fe-4c30-acc3-6ed32d9cd190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:20.038932 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23da3568-52fe-4c30-acc3-6ed32d9cd190 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:20.039210 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23da3568-52fe-4c30-acc3-6ed32d9cd190 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:20.039483 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23da3568-52fe-4c30-acc3-6ed32d9cd190 admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:20.040201 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23da3568-52fe-4c30-acc3-6ed32d9cd190 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:20.041829 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23da3568-52fe-4c30-acc3-6ed32d9cd190 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:20.051806 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1405/2785] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:56:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:56:20.056588 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd195c13-2b07-4fbf-af9f-82b5ad560e0e admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:20.066894 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd195c13-2b07-4fbf-af9f-82b5ad560e0e admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:20.072231 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0798b02e-78cb-4fe0-960a-d1b75e4bf0d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:20.072888 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0798b02e-78cb-4fe0-960a-d1b75e4bf0d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:20.073199 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0798b02e-78cb-4fe0-960a-d1b75e4bf0d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:20.073625 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0798b02e-78cb-4fe0-960a-d1b75e4bf0d1 admin admin] PATH_INFO: `/v3/projects/0313006ca1fa4ba4a293fa0d7dd629b7/users/6f9b63ec6fe24aa1811d875d4335fdac/roles/73e457231a824ea98c28d22610be640c` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:20.088388 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0798b02e-78cb-4fe0-960a-d1b75e4bf0d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0313006ca1fa4ba4a293fa0d7dd629b7, user_id=6f9b63ec6fe24aa1811d875d4335fdac, role_id=73e457231a824ea98c28d22610be640c)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:20.091081 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0798b02e-78cb-4fe0-960a-d1b75e4bf0d1 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:20.105838 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1381/2786] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:56:20 2022] PUT /identity/v3/projects/0313006ca1fa4ba4a293fa0d7dd629b7/users/6f9b63ec6fe24aa1811d875d4335fdac/roles/73e457231a824ea98c28d22610be640c => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:56:20.110908 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23da3568-52fe-4c30-acc3-6ed32d9cd190 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:20.122017 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23da3568-52fe-4c30-acc3-6ed32d9cd190 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:20.194751 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e59bca-3bc9-4101-bc13-ae45ccc7f106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:20.195378 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e59bca-3bc9-4101-bc13-ae45ccc7f106 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:20.195658 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e59bca-3bc9-4101-bc13-ae45ccc7f106 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:20.195933 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e59bca-3bc9-4101-bc13-ae45ccc7f106 admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:20.196685 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e59bca-3bc9-4101-bc13-ae45ccc7f106 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:20.198312 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e59bca-3bc9-4101-bc13-ae45ccc7f106 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:20.210795 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1406/2787] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:56:20 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:56:20.214920 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0798b02e-78cb-4fe0-960a-d1b75e4bf0d1 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:20.225792 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0798b02e-78cb-4fe0-960a-d1b75e4bf0d1 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:20.230401 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f5c800-a50d-48e2-a814-94c3003911ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:20.231070 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f5c800-a50d-48e2-a814-94c3003911ac admin admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:20.231353 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f5c800-a50d-48e2-a814-94c3003911ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:20.231624 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f5c800-a50d-48e2-a814-94c3003911ac admin admin] PATH_INFO: `/v3/projects/0313006ca1fa4ba4a293fa0d7dd629b7/users/6f9b63ec6fe24aa1811d875d4335fdac/roles/73e457231a824ea98c28d22610be640c` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:20.233675 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f5c800-a50d-48e2-a814-94c3003911ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=0313006ca1fa4ba4a293fa0d7dd629b7, user_id=6f9b63ec6fe24aa1811d875d4335fdac, role_id=73e457231a824ea98c28d22610be640c)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:20.236323 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f5c800-a50d-48e2-a814-94c3003911ac admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:20.247858 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1382/2788] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:56:20 2022] PUT /identity/v3/projects/0313006ca1fa4ba4a293fa0d7dd629b7/users/6f9b63ec6fe24aa1811d875d4335fdac/roles/73e457231a824ea98c28d22610be640c => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:56:20.259477 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b533bc5f-1a66-48e5-9286-230914edf2d8 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:20.259766 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b533bc5f-1a66-48e5-9286-230914edf2d8 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:20.260044 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b533bc5f-1a66-48e5-9286-230914edf2d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:20.289842 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-b533bc5f-1a66-48e5-9286-230914edf2d8 None None] MFA Rules not processed for user `6f9b63ec6fe24aa1811d875d4335fdac`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:56:20.291088 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-b533bc5f-1a66-48e5-9286-230914edf2d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:56:20.374197 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1407/2789] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:56:20 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:56:20.385528 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f5c800-a50d-48e2-a814-94c3003911ac admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:20.387317 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-f9f5c800-a50d-48e2-a814-94c3003911ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:56:20.455552 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f5c800-a50d-48e2-a814-94c3003911ac admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:20.459475 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d220c5-1ce0-45c9-9178-6af789eebb71 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96d512d0c84240918dae6c4a3e8b7b79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:20.460100 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d220c5-1ce0-45c9-9178-6af789eebb71 service cinder] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:20.460385 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d220c5-1ce0-45c9-9178-6af789eebb71 service cinder] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:20.460667 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d220c5-1ce0-45c9-9178-6af789eebb71 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:20.471801 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d220c5-1ce0-45c9-9178-6af789eebb71 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:20.473272 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d220c5-1ce0-45c9-9178-6af789eebb71 service cinder] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:20.485487 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1383/2790] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:56:20 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:56:20.515905 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b533bc5f-1a66-48e5-9286-230914edf2d8 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:20.578309 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:56:20.585439 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b533bc5f-1a66-48e5-9286-230914edf2d8 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:20.590477 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e7b571-09e5-43ec-b63f-e016320cc382 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:20.591111 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e7b571-09e5-43ec-b63f-e016320cc382 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:20.591393 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e7b571-09e5-43ec-b63f-e016320cc382 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:20.591667 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e7b571-09e5-43ec-b63f-e016320cc382 admin admin] PATH_INFO: `/v3/users/07f083cf9fae4837afd2ab90fe2cadff` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:20.592853 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e7b571-09e5-43ec-b63f-e016320cc382 admin admin] RBAC: Authorizing `identity:delete_user(user_id=07f083cf9fae4837afd2ab90fe2cadff)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:20.594830 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e7b571-09e5-43ec-b63f-e016320cc382 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:20.650175 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-f9e7b571-09e5-43ec-b63f-e016320cc382 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07f083cf9fae4837afd2ab90fe2cadff'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:56:20.658137 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-f9e7b571-09e5-43ec-b63f-e016320cc382 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07f083cf9fae4837afd2ab90fe2cadff'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:56:20.658831 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-f9e7b571-09e5-43ec-b63f-e016320cc382 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07f083cf9fae4837afd2ab90fe2cadff'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:56:20.667285 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-f9e7b571-09e5-43ec-b63f-e016320cc382 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07f083cf9fae4837afd2ab90fe2cadff'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:56:20.680120 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-f9e7b571-09e5-43ec-b63f-e016320cc382 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07f083cf9fae4837afd2ab90fe2cadff'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:56:20.686090 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1408/2791] 10.222.0.89 () {58 vars in 1224 bytes} [Thu Jul 28 17:56:20 2022] DELETE /identity/v3/users/07f083cf9fae4837afd2ab90fe2cadff => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:56:20.690766 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bf76f6-0b07-4346-9c51-63438a71a5fe None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:20.691054 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bf76f6-0b07-4346-9c51-63438a71a5fe None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:20.691333 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bf76f6-0b07-4346-9c51-63438a71a5fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:20.713548 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-25bf76f6-0b07-4346-9c51-63438a71a5fe None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:56:20.715000 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-25bf76f6-0b07-4346-9c51-63438a71a5fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:56:20.777790 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1384/2792] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:56:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:56:20.789023 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e7b571-09e5-43ec-b63f-e016320cc382 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:20.790524 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-f9e7b571-09e5-43ec-b63f-e016320cc382 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:56:20.985402 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e7b571-09e5-43ec-b63f-e016320cc382 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:20.988976 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6ee1d7-e523-4a66-9fde-edbcf3326613 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76817cd541294331b0023069092e8e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:20.989619 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6ee1d7-e523-4a66-9fde-edbcf3326613 service neutron] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:20.989898 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6ee1d7-e523-4a66-9fde-edbcf3326613 service neutron] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:20.990801 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6ee1d7-e523-4a66-9fde-edbcf3326613 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:21.000481 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6ee1d7-e523-4a66-9fde-edbcf3326613 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:21.001925 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6ee1d7-e523-4a66-9fde-edbcf3326613 service neutron] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:21.011517 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1409/2793] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:56:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:56:21.279612 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bf76f6-0b07-4346-9c51-63438a71a5fe None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:21.281108 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-25bf76f6-0b07-4346-9c51-63438a71a5fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:56:21.295828 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bf76f6-0b07-4346-9c51-63438a71a5fe None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:21.300653 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79c9dd4-004b-4844-9903-cbc2e6b90770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:21.301337 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79c9dd4-004b-4844-9903-cbc2e6b90770 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:21.301778 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79c9dd4-004b-4844-9903-cbc2e6b90770 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:21.302061 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79c9dd4-004b-4844-9903-cbc2e6b90770 admin admin] PATH_INFO: `/v3/projects/e55fb51bd40f4a10b03f33a34acf3bd9` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:21.304074 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79c9dd4-004b-4844-9903-cbc2e6b90770 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e55fb51bd40f4a10b03f33a34acf3bd9)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:21.306010 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79c9dd4-004b-4844-9903-cbc2e6b90770 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:21.350296 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:56:21.354273 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-d79c9dd4-004b-4844-9903-cbc2e6b90770 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e55fb51bd40f4a10b03f33a34acf3bd9', 'request_id': 'req-d79c9dd4-004b-4844-9903-cbc2e6b90770'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:56:21.361563 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-d79c9dd4-004b-4844-9903-cbc2e6b90770 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e55fb51bd40f4a10b03f33a34acf3bd9', 'request_id': 'req-d79c9dd4-004b-4844-9903-cbc2e6b90770'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:56:21.367552 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-d79c9dd4-004b-4844-9903-cbc2e6b90770 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e55fb51bd40f4a10b03f33a34acf3bd9', 'request_id': 'req-d79c9dd4-004b-4844-9903-cbc2e6b90770'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:56:21.379784 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-d79c9dd4-004b-4844-9903-cbc2e6b90770 admin admin] The token cache is being invalidate because project e55fb51bd40f4a10b03f33a34acf3bd9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80325) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 17:56:21.380067 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-d79c9dd4-004b-4844-9903-cbc2e6b90770 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:56:21.380901 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-d79c9dd4-004b-4844-9903-cbc2e6b90770 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:56:21.382639 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1385/2794] 10.222.0.89 () {58 vars in 1233 bytes} [Thu Jul 28 17:56:21 2022] DELETE /identity/v3/projects/e55fb51bd40f4a10b03f33a34acf3bd9 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:56:21.386746 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6ee1d7-e523-4a66-9fde-edbcf3326613 service neutron] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:21.396748 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6ee1d7-e523-4a66-9fde-edbcf3326613 service neutron] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:21.454591 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c428f00-5348-4a15-ba0b-2d7c06cc4499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:21.455211 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c428f00-5348-4a15-ba0b-2d7c06cc4499 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:21.455486 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c428f00-5348-4a15-ba0b-2d7c06cc4499 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:21.455755 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c428f00-5348-4a15-ba0b-2d7c06cc4499 admin admin] PATH_INFO: `/v3/users/6f9b63ec6fe24aa1811d875d4335fdac` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:21.456909 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c428f00-5348-4a15-ba0b-2d7c06cc4499 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f9b63ec6fe24aa1811d875d4335fdac)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:21.459820 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c428f00-5348-4a15-ba0b-2d7c06cc4499 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:21.481344 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:56:21.507420 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-8c428f00-5348-4a15-ba0b-2d7c06cc4499 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f9b63ec6fe24aa1811d875d4335fdac'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:56:21.514237 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-8c428f00-5348-4a15-ba0b-2d7c06cc4499 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f9b63ec6fe24aa1811d875d4335fdac'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:56:21.514938 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-8c428f00-5348-4a15-ba0b-2d7c06cc4499 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f9b63ec6fe24aa1811d875d4335fdac'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:56:21.523116 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-8c428f00-5348-4a15-ba0b-2d7c06cc4499 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f9b63ec6fe24aa1811d875d4335fdac'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:56:21.534910 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-8c428f00-5348-4a15-ba0b-2d7c06cc4499 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f9b63ec6fe24aa1811d875d4335fdac'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:56:21.542193 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1410/2795] 10.222.0.89 () {58 vars in 1224 bytes} [Thu Jul 28 17:56:21 2022] DELETE /identity/v3/users/6f9b63ec6fe24aa1811d875d4335fdac => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 17:56:21.828620 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79c9dd4-004b-4844-9903-cbc2e6b90770 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:21.831305 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-d79c9dd4-004b-4844-9903-cbc2e6b90770 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:56:21.845787 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79c9dd4-004b-4844-9903-cbc2e6b90770 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:21.903959 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1631f298-2567-44a8-96fd-dc000251ed56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:21.904610 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1631f298-2567-44a8-96fd-dc000251ed56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:21.904890 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1631f298-2567-44a8-96fd-dc000251ed56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:21.905176 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1631f298-2567-44a8-96fd-dc000251ed56 admin admin] PATH_INFO: `/v3/projects/0313006ca1fa4ba4a293fa0d7dd629b7` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:21.906382 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1631f298-2567-44a8-96fd-dc000251ed56 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0313006ca1fa4ba4a293fa0d7dd629b7)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:21.908279 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1631f298-2567-44a8-96fd-dc000251ed56 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:21.955248 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-1631f298-2567-44a8-96fd-dc000251ed56 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0313006ca1fa4ba4a293fa0d7dd629b7', 'request_id': 'req-1631f298-2567-44a8-96fd-dc000251ed56'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:56:21.961939 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-1631f298-2567-44a8-96fd-dc000251ed56 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0313006ca1fa4ba4a293fa0d7dd629b7', 'request_id': 'req-1631f298-2567-44a8-96fd-dc000251ed56'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:56:21.967638 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-1631f298-2567-44a8-96fd-dc000251ed56 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0313006ca1fa4ba4a293fa0d7dd629b7', 'request_id': 'req-1631f298-2567-44a8-96fd-dc000251ed56'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:56:21.981229 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-1631f298-2567-44a8-96fd-dc000251ed56 admin admin] The token cache is being invalidate because project 0313006ca1fa4ba4a293fa0d7dd629b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80325) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 17:56:21.981584 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-1631f298-2567-44a8-96fd-dc000251ed56 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:56:21.982231 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-1631f298-2567-44a8-96fd-dc000251ed56 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:56:21.983737 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1386/2796] 10.222.0.89 () {58 vars in 1233 bytes} [Thu Jul 28 17:56:21 2022] DELETE /identity/v3/projects/0313006ca1fa4ba4a293fa0d7dd629b7 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:56:21.988576 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de385228-cfda-4f04-84a4-0dde642024b5 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:21.989099 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de385228-cfda-4f04-84a4-0dde642024b5 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:21.989232 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de385228-cfda-4f04-84a4-0dde642024b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:22.011171 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-de385228-cfda-4f04-84a4-0dde642024b5 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:56:22.012672 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-de385228-cfda-4f04-84a4-0dde642024b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:56:22.071983 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1411/2797] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:56:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:56:22.079559 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d63bab7-7def-4909-8649-e25c7bd1c84c None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:22.079849 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d63bab7-7def-4909-8649-e25c7bd1c84c None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:22.080133 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d63bab7-7def-4909-8649-e25c7bd1c84c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:22.101184 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-1d63bab7-7def-4909-8649-e25c7bd1c84c None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:56:22.102420 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-1d63bab7-7def-4909-8649-e25c7bd1c84c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:56:22.111459 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1387/2798] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:56:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:56:22.116130 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de385228-cfda-4f04-84a4-0dde642024b5 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:22.126312 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de385228-cfda-4f04-84a4-0dde642024b5 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:22.131527 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e87863-60cd-41ae-9a85-ee65a3c73891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:22.132131 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e87863-60cd-41ae-9a85-ee65a3c73891 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:22.132412 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e87863-60cd-41ae-9a85-ee65a3c73891 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:22.132685 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e87863-60cd-41ae-9a85-ee65a3c73891 admin admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:22.133413 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e87863-60cd-41ae-9a85-ee65a3c73891 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:56:22.133719 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e87863-60cd-41ae-9a85-ee65a3c73891 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:22.135319 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e87863-60cd-41ae-9a85-ee65a3c73891 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:22.146779 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1412/2799] 10.222.0.89 () {58 vars in 1153 bytes} [Thu Jul 28 17:56:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:56:22.151261 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d63bab7-7def-4909-8649-e25c7bd1c84c None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:22.161029 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d63bab7-7def-4909-8649-e25c7bd1c84c None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:22.166034 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed493b1d-9983-46db-b8b8-66e930869bf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:22.166688 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed493b1d-9983-46db-b8b8-66e930869bf4 admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:22.166970 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed493b1d-9983-46db-b8b8-66e930869bf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:22.167269 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed493b1d-9983-46db-b8b8-66e930869bf4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:22.167956 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed493b1d-9983-46db-b8b8-66e930869bf4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:22.169932 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed493b1d-9983-46db-b8b8-66e930869bf4 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:22.198530 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1388/2800] 10.222.0.89 () {60 vars in 1153 bytes} [Thu Jul 28 17:56:22 2022] POST /identity/v3/projects => generated 381 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 17:56:22.202737 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e87863-60cd-41ae-9a85-ee65a3c73891 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:22.213513 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e87863-60cd-41ae-9a85-ee65a3c73891 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:22.269258 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff796ad-6fa5-4bc6-9c6f-1185ed3f900c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:22.269869 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff796ad-6fa5-4bc6-9c6f-1185ed3f900c admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:22.270149 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff796ad-6fa5-4bc6-9c6f-1185ed3f900c admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:22.270421 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff796ad-6fa5-4bc6-9c6f-1185ed3f900c admin admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:22.271090 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff796ad-6fa5-4bc6-9c6f-1185ed3f900c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:22.273178 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff796ad-6fa5-4bc6-9c6f-1185ed3f900c admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:22.294540 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1413/2801] 10.222.0.89 () {60 vars in 1144 bytes} [Thu Jul 28 17:56:22 2022] POST /identity/v3/users => generated 291 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:56:22.298742 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed493b1d-9983-46db-b8b8-66e930869bf4 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:22.310205 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed493b1d-9983-46db-b8b8-66e930869bf4 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:22.315197 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79894a6c-0ed4-44c3-a6c5-66947546a793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:22.315829 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79894a6c-0ed4-44c3-a6c5-66947546a793 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:22.316111 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79894a6c-0ed4-44c3-a6c5-66947546a793 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:22.316384 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79894a6c-0ed4-44c3-a6c5-66947546a793 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:22.317077 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79894a6c-0ed4-44c3-a6c5-66947546a793 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:22.319290 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79894a6c-0ed4-44c3-a6c5-66947546a793 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:22.329312 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1389/2802] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:56:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:56:22.333545 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff796ad-6fa5-4bc6-9c6f-1185ed3f900c admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:22.345024 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff796ad-6fa5-4bc6-9c6f-1185ed3f900c admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:22.349893 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aad8746-3348-4922-8e8a-d5b9338cd323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:22.350429 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aad8746-3348-4922-8e8a-d5b9338cd323 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:22.350712 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aad8746-3348-4922-8e8a-d5b9338cd323 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:22.351014 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aad8746-3348-4922-8e8a-d5b9338cd323 admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:22.351706 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aad8746-3348-4922-8e8a-d5b9338cd323 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:22.353362 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aad8746-3348-4922-8e8a-d5b9338cd323 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:22.363414 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1414/2803] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:56:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:56:22.368355 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79894a6c-0ed4-44c3-a6c5-66947546a793 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:22.378438 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79894a6c-0ed4-44c3-a6c5-66947546a793 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:22.383778 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77aace0b-c9e9-4af1-9dfc-e4deeed11bfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:22.384433 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aace0b-c9e9-4af1-9dfc-e4deeed11bfe admin admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:22.384712 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aace0b-c9e9-4af1-9dfc-e4deeed11bfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:22.384986 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aace0b-c9e9-4af1-9dfc-e4deeed11bfe admin admin] PATH_INFO: `/v3/projects/a851547b80e84fd1977ca5d148825e16/users/2c1327c8f8de4f5da04afb38d7920335/roles/11f8731756fe4caca13efa61b71b11c5` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:22.399658 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77aace0b-c9e9-4af1-9dfc-e4deeed11bfe admin admin] RBAC: Authorizing `identity:create_grant(project_id=a851547b80e84fd1977ca5d148825e16, user_id=2c1327c8f8de4f5da04afb38d7920335, role_id=11f8731756fe4caca13efa61b71b11c5)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:22.402623 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77aace0b-c9e9-4af1-9dfc-e4deeed11bfe admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:22.414416 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:56:22.422326 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1390/2804] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:56:22 2022] PUT /identity/v3/projects/a851547b80e84fd1977ca5d148825e16/users/2c1327c8f8de4f5da04afb38d7920335/roles/11f8731756fe4caca13efa61b71b11c5 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:56:22.433141 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0803f0-f60e-4e3a-840a-672e62800073 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:22.433451 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0803f0-f60e-4e3a-840a-672e62800073 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:22.433776 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0803f0-f60e-4e3a-840a-672e62800073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:22.441109 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:DELETE=8,keystone:SELECT=88,keystone:INSERT=4 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:56:22.464754 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-bb0803f0-f60e-4e3a-840a-672e62800073 None None] MFA Rules not processed for user `2c1327c8f8de4f5da04afb38d7920335`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:56:22.466372 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-bb0803f0-f60e-4e3a-840a-672e62800073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:56:22.531040 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1415/2805] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:56:22 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:56:22.535556 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77aace0b-c9e9-4af1-9dfc-e4deeed11bfe admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:22.598156 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77aace0b-c9e9-4af1-9dfc-e4deeed11bfe admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:22.603506 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453a7752-7474-4e9b-a0e6-61568890c49d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:22.604124 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453a7752-7474-4e9b-a0e6-61568890c49d admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:22.604406 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453a7752-7474-4e9b-a0e6-61568890c49d admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:22.604682 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453a7752-7474-4e9b-a0e6-61568890c49d admin admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:22.605394 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453a7752-7474-4e9b-a0e6-61568890c49d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:22.607287 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453a7752-7474-4e9b-a0e6-61568890c49d admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:22.634654 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1391/2806] 10.222.0.89 () {60 vars in 1153 bytes} [Thu Jul 28 17:56:22 2022] POST /identity/v3/projects => generated 381 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:56:22.638746 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0803f0-f60e-4e3a-840a-672e62800073 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:22.649932 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0803f0-f60e-4e3a-840a-672e62800073 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:22.706478 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9cbabf-9ac1-49c2-957c-17fb7b0f93ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:22.707088 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9cbabf-9ac1-49c2-957c-17fb7b0f93ca admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:22.707365 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9cbabf-9ac1-49c2-957c-17fb7b0f93ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:22.707636 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9cbabf-9ac1-49c2-957c-17fb7b0f93ca admin admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:22.708289 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9cbabf-9ac1-49c2-957c-17fb7b0f93ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:22.710721 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9cbabf-9ac1-49c2-957c-17fb7b0f93ca admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:22.732081 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1416/2807] 10.222.0.89 () {60 vars in 1144 bytes} [Thu Jul 28 17:56:22 2022] POST /identity/v3/users => generated 297 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:56:22.736103 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453a7752-7474-4e9b-a0e6-61568890c49d admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:22.747135 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453a7752-7474-4e9b-a0e6-61568890c49d admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:22.751880 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb44411-02e7-4dff-8be9-fe417eac5eb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:22.752485 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb44411-02e7-4dff-8be9-fe417eac5eb8 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:22.752770 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb44411-02e7-4dff-8be9-fe417eac5eb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:22.753048 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb44411-02e7-4dff-8be9-fe417eac5eb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:22.753795 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb44411-02e7-4dff-8be9-fe417eac5eb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:22.755368 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb44411-02e7-4dff-8be9-fe417eac5eb8 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:22.765970 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1392/2808] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:56:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:56:22.770248 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9cbabf-9ac1-49c2-957c-17fb7b0f93ca admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:22.781122 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9cbabf-9ac1-49c2-957c-17fb7b0f93ca admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:22.785803 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76433117-3c91-463b-93f7-fd37dead5044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:22.786449 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76433117-3c91-463b-93f7-fd37dead5044 admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:22.787233 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76433117-3c91-463b-93f7-fd37dead5044 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:22.788091 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76433117-3c91-463b-93f7-fd37dead5044 admin admin] PATH_INFO: `/v3/projects/b99edabeefb14f59ade2e81d2670a121/users/97f023b1c89341908681d8b040e439ef/roles/73e457231a824ea98c28d22610be640c` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:22.803903 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76433117-3c91-463b-93f7-fd37dead5044 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b99edabeefb14f59ade2e81d2670a121, user_id=97f023b1c89341908681d8b040e439ef, role_id=73e457231a824ea98c28d22610be640c)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:22.806658 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76433117-3c91-463b-93f7-fd37dead5044 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:22.821667 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1417/2809] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:56:22 2022] PUT /identity/v3/projects/b99edabeefb14f59ade2e81d2670a121/users/97f023b1c89341908681d8b040e439ef/roles/73e457231a824ea98c28d22610be640c => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 17:56:22.825664 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb44411-02e7-4dff-8be9-fe417eac5eb8 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:22.837126 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb44411-02e7-4dff-8be9-fe417eac5eb8 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:22.895100 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134a299b-525a-4ff2-b7a3-a247f9f2ea3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:22.895641 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134a299b-525a-4ff2-b7a3-a247f9f2ea3c admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:22.895923 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134a299b-525a-4ff2-b7a3-a247f9f2ea3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:22.896727 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134a299b-525a-4ff2-b7a3-a247f9f2ea3c admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:22.897552 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134a299b-525a-4ff2-b7a3-a247f9f2ea3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:22.899232 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134a299b-525a-4ff2-b7a3-a247f9f2ea3c admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:22.909699 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1393/2810] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:56:22 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 17:56:22.914466 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76433117-3c91-463b-93f7-fd37dead5044 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:22.924993 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76433117-3c91-463b-93f7-fd37dead5044 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:22.929809 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42134cb2-51b4-4191-a19c-a1ae62d1d7f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:22.930459 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42134cb2-51b4-4191-a19c-a1ae62d1d7f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:22.930739 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42134cb2-51b4-4191-a19c-a1ae62d1d7f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:22.931013 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42134cb2-51b4-4191-a19c-a1ae62d1d7f7 admin admin] PATH_INFO: `/v3/projects/b99edabeefb14f59ade2e81d2670a121/users/97f023b1c89341908681d8b040e439ef/roles/73e457231a824ea98c28d22610be640c` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:22.933034 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42134cb2-51b4-4191-a19c-a1ae62d1d7f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b99edabeefb14f59ade2e81d2670a121, user_id=97f023b1c89341908681d8b040e439ef, role_id=73e457231a824ea98c28d22610be640c)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:22.936000 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42134cb2-51b4-4191-a19c-a1ae62d1d7f7 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:22.946266 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1418/2811] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:56:22 2022] PUT /identity/v3/projects/b99edabeefb14f59ade2e81d2670a121/users/97f023b1c89341908681d8b040e439ef/roles/73e457231a824ea98c28d22610be640c => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:56:22.957634 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1600ac-a133-47bd-bde7-c3ca1e602bf4 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:22.957926 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1600ac-a133-47bd-bde7-c3ca1e602bf4 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:22.958208 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1600ac-a133-47bd-bde7-c3ca1e602bf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:22.987915 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-7e1600ac-a133-47bd-bde7-c3ca1e602bf4 None None] MFA Rules not processed for user `97f023b1c89341908681d8b040e439ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:56:22.989141 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-7e1600ac-a133-47bd-bde7-c3ca1e602bf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:56:23.047780 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:56:23.059868 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1394/2812] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:56:22 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:56:23.072139 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42134cb2-51b4-4191-a19c-a1ae62d1d7f7 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:23.073920 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-42134cb2-51b4-4191-a19c-a1ae62d1d7f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:56:23.122658 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:56:23.135154 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42134cb2-51b4-4191-a19c-a1ae62d1d7f7 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:23.138849 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09429adc-a1b3-4de7-b823-49183df0847e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96d512d0c84240918dae6c4a3e8b7b79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:23.139460 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09429adc-a1b3-4de7-b823-49183df0847e service cinder] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:23.139743 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09429adc-a1b3-4de7-b823-49183df0847e service cinder] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:23.140015 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09429adc-a1b3-4de7-b823-49183df0847e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:23.150157 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09429adc-a1b3-4de7-b823-49183df0847e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:23.151812 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09429adc-a1b3-4de7-b823-49183df0847e service cinder] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:23.161560 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1419/2813] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:56:23 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:56:23.470845 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1600ac-a133-47bd-bde7-c3ca1e602bf4 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:23.480596 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1600ac-a133-47bd-bde7-c3ca1e602bf4 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:23.485104 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58b4d53-46c6-436a-b6b5-42ee3976b418 tempest-VolumeQuotaClassesTest-1012514807 tempest-VolumeQuotaClassesTest-1012514807-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97f023b1c89341908681d8b040e439ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b99edabeefb14f59ade2e81d2670a121', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:23.485744 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58b4d53-46c6-436a-b6b5-42ee3976b418 tempest-VolumeQuotaClassesTest-1012514807 tempest-VolumeQuotaClassesTest-1012514807-project-admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:23.486023 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58b4d53-46c6-436a-b6b5-42ee3976b418 tempest-VolumeQuotaClassesTest-1012514807 tempest-VolumeQuotaClassesTest-1012514807-project-admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:23.486296 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58b4d53-46c6-436a-b6b5-42ee3976b418 tempest-VolumeQuotaClassesTest-1012514807 tempest-VolumeQuotaClassesTest-1012514807-project-admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:23.486985 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58b4d53-46c6-436a-b6b5-42ee3976b418 tempest-VolumeQuotaClassesTest-1012514807 tempest-VolumeQuotaClassesTest-1012514807-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:56:23.487289 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58b4d53-46c6-436a-b6b5-42ee3976b418 tempest-VolumeQuotaClassesTest-1012514807 tempest-VolumeQuotaClassesTest-1012514807-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:23.488888 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58b4d53-46c6-436a-b6b5-42ee3976b418 tempest-VolumeQuotaClassesTest-1012514807 tempest-VolumeQuotaClassesTest-1012514807-project-admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:23.501002 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1395/2814] 10.222.0.89 () {58 vars in 1153 bytes} [Thu Jul 28 17:56:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:56:23.505105 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09429adc-a1b3-4de7-b823-49183df0847e service cinder] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:23.514784 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09429adc-a1b3-4de7-b823-49183df0847e service cinder] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:23.519525 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018321d5-837a-40af-a6ca-65cffb02760a tempest-VolumeQuotaClassesTest-1012514807 tempest-VolumeQuotaClassesTest-1012514807-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97f023b1c89341908681d8b040e439ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b99edabeefb14f59ade2e81d2670a121', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:23.520122 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018321d5-837a-40af-a6ca-65cffb02760a tempest-VolumeQuotaClassesTest-1012514807 tempest-VolumeQuotaClassesTest-1012514807-project-admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:23.520387 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018321d5-837a-40af-a6ca-65cffb02760a tempest-VolumeQuotaClassesTest-1012514807 tempest-VolumeQuotaClassesTest-1012514807-project-admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:23.520656 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018321d5-837a-40af-a6ca-65cffb02760a tempest-VolumeQuotaClassesTest-1012514807 tempest-VolumeQuotaClassesTest-1012514807-project-admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:23.521869 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018321d5-837a-40af-a6ca-65cffb02760a tempest-VolumeQuotaClassesTest-1012514807 tempest-VolumeQuotaClassesTest-1012514807-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:23.523734 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018321d5-837a-40af-a6ca-65cffb02760a tempest-VolumeQuotaClassesTest-1012514807 tempest-VolumeQuotaClassesTest-1012514807-project-admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:23.550967 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1420/2815] 10.222.0.89 () {60 vars in 1153 bytes} [Thu Jul 28 17:56:23 2022] POST /identity/v3/projects => generated 354 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:56:23.555725 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58b4d53-46c6-436a-b6b5-42ee3976b418 tempest-VolumeQuotaClassesTest-1012514807 tempest-VolumeQuotaClassesTest-1012514807-project-admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:23.565631 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58b4d53-46c6-436a-b6b5-42ee3976b418 tempest-VolumeQuotaClassesTest-1012514807 tempest-VolumeQuotaClassesTest-1012514807-project-admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:23.623960 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bb4479-41e0-4fb9-bca0-92406f4f0e3e tempest-VolumeQuotaClassesTest-1012514807 tempest-VolumeQuotaClassesTest-1012514807-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97f023b1c89341908681d8b040e439ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b99edabeefb14f59ade2e81d2670a121', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:23.624578 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bb4479-41e0-4fb9-bca0-92406f4f0e3e tempest-VolumeQuotaClassesTest-1012514807 tempest-VolumeQuotaClassesTest-1012514807-project-admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:23.624863 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bb4479-41e0-4fb9-bca0-92406f4f0e3e tempest-VolumeQuotaClassesTest-1012514807 tempest-VolumeQuotaClassesTest-1012514807-project-admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:23.625140 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bb4479-41e0-4fb9-bca0-92406f4f0e3e tempest-VolumeQuotaClassesTest-1012514807 tempest-VolumeQuotaClassesTest-1012514807-project-admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:23.625866 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bb4479-41e0-4fb9-bca0-92406f4f0e3e tempest-VolumeQuotaClassesTest-1012514807 tempest-VolumeQuotaClassesTest-1012514807-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:56:23.626174 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bb4479-41e0-4fb9-bca0-92406f4f0e3e tempest-VolumeQuotaClassesTest-1012514807 tempest-VolumeQuotaClassesTest-1012514807-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:23.627790 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bb4479-41e0-4fb9-bca0-92406f4f0e3e tempest-VolumeQuotaClassesTest-1012514807 tempest-VolumeQuotaClassesTest-1012514807-project-admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:23.640758 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1396/2816] 10.222.0.89 () {58 vars in 1153 bytes} [Thu Jul 28 17:56:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:56:23.680064 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018321d5-837a-40af-a6ca-65cffb02760a tempest-VolumeQuotaClassesTest-1012514807 tempest-VolumeQuotaClassesTest-1012514807-project-admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:23.690413 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018321d5-837a-40af-a6ca-65cffb02760a tempest-VolumeQuotaClassesTest-1012514807 tempest-VolumeQuotaClassesTest-1012514807-project-admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:23.696056 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de826e77-eebd-41ac-8afc-6685f3f00403 tempest-VolumeQuotaClassesTest-1012514807 tempest-VolumeQuotaClassesTest-1012514807-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97f023b1c89341908681d8b040e439ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b99edabeefb14f59ade2e81d2670a121', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:23.696748 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de826e77-eebd-41ac-8afc-6685f3f00403 tempest-VolumeQuotaClassesTest-1012514807 tempest-VolumeQuotaClassesTest-1012514807-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:23.697033 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de826e77-eebd-41ac-8afc-6685f3f00403 tempest-VolumeQuotaClassesTest-1012514807 tempest-VolumeQuotaClassesTest-1012514807-project-admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:23.697357 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de826e77-eebd-41ac-8afc-6685f3f00403 tempest-VolumeQuotaClassesTest-1012514807 tempest-VolumeQuotaClassesTest-1012514807-project-admin] PATH_INFO: `/v3/projects/69a028599803488d9cd1dd64b79cea40` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:23.698540 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de826e77-eebd-41ac-8afc-6685f3f00403 tempest-VolumeQuotaClassesTest-1012514807 tempest-VolumeQuotaClassesTest-1012514807-project-admin] RBAC: Authorizing `identity:delete_project(project_id=69a028599803488d9cd1dd64b79cea40)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:23.700458 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de826e77-eebd-41ac-8afc-6685f3f00403 tempest-VolumeQuotaClassesTest-1012514807 tempest-VolumeQuotaClassesTest-1012514807-project-admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:23.747174 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-de826e77-eebd-41ac-8afc-6685f3f00403 tempest-VolumeQuotaClassesTest-1012514807 tempest-VolumeQuotaClassesTest-1012514807-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69a028599803488d9cd1dd64b79cea40', 'request_id': 'req-de826e77-eebd-41ac-8afc-6685f3f00403'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:56:23.752667 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-de826e77-eebd-41ac-8afc-6685f3f00403 tempest-VolumeQuotaClassesTest-1012514807 tempest-VolumeQuotaClassesTest-1012514807-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69a028599803488d9cd1dd64b79cea40', 'request_id': 'req-de826e77-eebd-41ac-8afc-6685f3f00403'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:56:23.759322 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-de826e77-eebd-41ac-8afc-6685f3f00403 tempest-VolumeQuotaClassesTest-1012514807 tempest-VolumeQuotaClassesTest-1012514807-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69a028599803488d9cd1dd64b79cea40', 'request_id': 'req-de826e77-eebd-41ac-8afc-6685f3f00403'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:56:23.773146 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-de826e77-eebd-41ac-8afc-6685f3f00403 tempest-VolumeQuotaClassesTest-1012514807 tempest-VolumeQuotaClassesTest-1012514807-project-admin] The token cache is being invalidate because project 69a028599803488d9cd1dd64b79cea40 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80324) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 17:56:23.773483 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-de826e77-eebd-41ac-8afc-6685f3f00403 tempest-VolumeQuotaClassesTest-1012514807 tempest-VolumeQuotaClassesTest-1012514807-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:56:23.774141 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-de826e77-eebd-41ac-8afc-6685f3f00403 tempest-VolumeQuotaClassesTest-1012514807 tempest-VolumeQuotaClassesTest-1012514807-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:56:23.775933 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1421/2817] 10.222.0.89 () {58 vars in 1233 bytes} [Thu Jul 28 17:56:23 2022] DELETE /identity/v3/projects/69a028599803488d9cd1dd64b79cea40 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:56:23.977589 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bb4479-41e0-4fb9-bca0-92406f4f0e3e tempest-VolumeQuotaClassesTest-1012514807 tempest-VolumeQuotaClassesTest-1012514807-project-admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:23.979376 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-03bb4479-41e0-4fb9-bca0-92406f4f0e3e tempest-VolumeQuotaClassesTest-1012514807 tempest-VolumeQuotaClassesTest-1012514807-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:56:24.046508 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bb4479-41e0-4fb9-bca0-92406f4f0e3e tempest-VolumeQuotaClassesTest-1012514807 tempest-VolumeQuotaClassesTest-1012514807-project-admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:24.103578 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f180eae-6576-48a3-a17b-424a87f52ead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:24.104216 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f180eae-6576-48a3-a17b-424a87f52ead admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:24.104498 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f180eae-6576-48a3-a17b-424a87f52ead admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:24.104774 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f180eae-6576-48a3-a17b-424a87f52ead admin admin] PATH_INFO: `/v3/users/2c1327c8f8de4f5da04afb38d7920335` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:24.106037 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f180eae-6576-48a3-a17b-424a87f52ead admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c1327c8f8de4f5da04afb38d7920335)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:24.107940 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f180eae-6576-48a3-a17b-424a87f52ead admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:24.129747 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:56:24.155751 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-8f180eae-6576-48a3-a17b-424a87f52ead admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c1327c8f8de4f5da04afb38d7920335'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:56:24.163174 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-8f180eae-6576-48a3-a17b-424a87f52ead admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c1327c8f8de4f5da04afb38d7920335'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:56:24.170938 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-8f180eae-6576-48a3-a17b-424a87f52ead admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c1327c8f8de4f5da04afb38d7920335'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:56:24.171583 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-8f180eae-6576-48a3-a17b-424a87f52ead admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c1327c8f8de4f5da04afb38d7920335'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:56:24.172199 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-8f180eae-6576-48a3-a17b-424a87f52ead admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c1327c8f8de4f5da04afb38d7920335'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:56:24.189206 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1397/2818] 10.222.0.89 () {58 vars in 1224 bytes} [Thu Jul 28 17:56:23 2022] DELETE /identity/v3/users/2c1327c8f8de4f5da04afb38d7920335 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:56:24.194185 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e69749-709f-4953-81e6-03a7df6dcc73 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:24.194695 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e69749-709f-4953-81e6-03a7df6dcc73 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:24.194808 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e69749-709f-4953-81e6-03a7df6dcc73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:24.213976 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-f4e69749-709f-4953-81e6-03a7df6dcc73 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:56:24.215203 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-f4e69749-709f-4953-81e6-03a7df6dcc73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:56:24.274117 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1422/2819] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:56:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:56:24.284291 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f180eae-6576-48a3-a17b-424a87f52ead admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:24.285738 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-8f180eae-6576-48a3-a17b-424a87f52ead admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:56:24.344568 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f180eae-6576-48a3-a17b-424a87f52ead admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:24.348517 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f586ca02-6331-4f7b-aa49-fdbc8b4b789d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76817cd541294331b0023069092e8e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:24.349139 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f586ca02-6331-4f7b-aa49-fdbc8b4b789d service neutron] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:24.349633 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f586ca02-6331-4f7b-aa49-fdbc8b4b789d service neutron] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:24.349633 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f586ca02-6331-4f7b-aa49-fdbc8b4b789d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:24.359197 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f586ca02-6331-4f7b-aa49-fdbc8b4b789d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:24.360620 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f586ca02-6331-4f7b-aa49-fdbc8b4b789d service neutron] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:24.370622 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1398/2820] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:56:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:56:24.642228 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e69749-709f-4953-81e6-03a7df6dcc73 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:24.643669 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-f4e69749-709f-4953-81e6-03a7df6dcc73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:56:24.657244 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e69749-709f-4953-81e6-03a7df6dcc73 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:24.662270 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d556d72e-f0bd-4790-ae3a-3688b9f26fbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:24.663112 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d556d72e-f0bd-4790-ae3a-3688b9f26fbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:24.663386 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d556d72e-f0bd-4790-ae3a-3688b9f26fbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:24.663651 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d556d72e-f0bd-4790-ae3a-3688b9f26fbf admin admin] PATH_INFO: `/v3/projects/a851547b80e84fd1977ca5d148825e16` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:24.664857 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d556d72e-f0bd-4790-ae3a-3688b9f26fbf admin admin] RBAC: Authorizing `identity:delete_project(project_id=a851547b80e84fd1977ca5d148825e16)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:24.666847 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d556d72e-f0bd-4790-ae3a-3688b9f26fbf admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:24.710865 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:56:24.715262 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-d556d72e-f0bd-4790-ae3a-3688b9f26fbf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a851547b80e84fd1977ca5d148825e16', 'request_id': 'req-d556d72e-f0bd-4790-ae3a-3688b9f26fbf'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:56:24.722557 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-d556d72e-f0bd-4790-ae3a-3688b9f26fbf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a851547b80e84fd1977ca5d148825e16', 'request_id': 'req-d556d72e-f0bd-4790-ae3a-3688b9f26fbf'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:56:24.729126 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-d556d72e-f0bd-4790-ae3a-3688b9f26fbf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a851547b80e84fd1977ca5d148825e16', 'request_id': 'req-d556d72e-f0bd-4790-ae3a-3688b9f26fbf'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:56:24.742355 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-d556d72e-f0bd-4790-ae3a-3688b9f26fbf admin admin] The token cache is being invalidate because project a851547b80e84fd1977ca5d148825e16 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80324) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 17:56:24.742674 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-d556d72e-f0bd-4790-ae3a-3688b9f26fbf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:56:24.743304 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-d556d72e-f0bd-4790-ae3a-3688b9f26fbf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:56:24.744825 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1423/2821] 10.222.0.89 () {58 vars in 1233 bytes} [Thu Jul 28 17:56:24 2022] DELETE /identity/v3/projects/a851547b80e84fd1977ca5d148825e16 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:56:24.748814 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f586ca02-6331-4f7b-aa49-fdbc8b4b789d service neutron] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:24.758566 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f586ca02-6331-4f7b-aa49-fdbc8b4b789d service neutron] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:24.816013 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf0e205-2138-4cd3-b01a-d1fdac864d8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:24.816663 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf0e205-2138-4cd3-b01a-d1fdac864d8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:24.816943 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf0e205-2138-4cd3-b01a-d1fdac864d8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:24.817258 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf0e205-2138-4cd3-b01a-d1fdac864d8e admin admin] PATH_INFO: `/v3/users/97f023b1c89341908681d8b040e439ef` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:24.818530 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf0e205-2138-4cd3-b01a-d1fdac864d8e admin admin] RBAC: Authorizing `identity:delete_user(user_id=97f023b1c89341908681d8b040e439ef)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:24.820436 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf0e205-2138-4cd3-b01a-d1fdac864d8e admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:24.865501 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:56:24.867459 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-3cf0e205-2138-4cd3-b01a-d1fdac864d8e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97f023b1c89341908681d8b040e439ef'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:56:24.876506 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-3cf0e205-2138-4cd3-b01a-d1fdac864d8e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97f023b1c89341908681d8b040e439ef'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:56:24.887422 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-3cf0e205-2138-4cd3-b01a-d1fdac864d8e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97f023b1c89341908681d8b040e439ef'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:56:24.888193 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-3cf0e205-2138-4cd3-b01a-d1fdac864d8e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97f023b1c89341908681d8b040e439ef'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:56:24.888681 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-3cf0e205-2138-4cd3-b01a-d1fdac864d8e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97f023b1c89341908681d8b040e439ef'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:56:24.906804 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1399/2822] 10.222.0.89 () {58 vars in 1224 bytes} [Thu Jul 28 17:56:24 2022] DELETE /identity/v3/users/97f023b1c89341908681d8b040e439ef => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:56:25.175095 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d556d72e-f0bd-4790-ae3a-3688b9f26fbf admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:25.176531 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-d556d72e-f0bd-4790-ae3a-3688b9f26fbf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:56:25.195150 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d556d72e-f0bd-4790-ae3a-3688b9f26fbf admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:25.253791 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a04400-c801-4cef-a161-06f5fbda3d8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:25.254341 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a04400-c801-4cef-a161-06f5fbda3d8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:25.254616 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a04400-c801-4cef-a161-06f5fbda3d8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:25.254886 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a04400-c801-4cef-a161-06f5fbda3d8b admin admin] PATH_INFO: `/v3/projects/b99edabeefb14f59ade2e81d2670a121` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:25.256354 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a04400-c801-4cef-a161-06f5fbda3d8b admin admin] RBAC: Authorizing `identity:delete_project(project_id=b99edabeefb14f59ade2e81d2670a121)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:25.258363 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a04400-c801-4cef-a161-06f5fbda3d8b admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:25.305185 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-83a04400-c801-4cef-a161-06f5fbda3d8b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b99edabeefb14f59ade2e81d2670a121', 'request_id': 'req-83a04400-c801-4cef-a161-06f5fbda3d8b'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:56:25.310577 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-83a04400-c801-4cef-a161-06f5fbda3d8b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b99edabeefb14f59ade2e81d2670a121', 'request_id': 'req-83a04400-c801-4cef-a161-06f5fbda3d8b'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:56:25.316670 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-83a04400-c801-4cef-a161-06f5fbda3d8b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b99edabeefb14f59ade2e81d2670a121', 'request_id': 'req-83a04400-c801-4cef-a161-06f5fbda3d8b'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:56:25.328606 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-83a04400-c801-4cef-a161-06f5fbda3d8b admin admin] The token cache is being invalidate because project b99edabeefb14f59ade2e81d2670a121 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80324) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 17:56:25.328948 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-83a04400-c801-4cef-a161-06f5fbda3d8b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:56:25.329595 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-83a04400-c801-4cef-a161-06f5fbda3d8b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:56:25.331094 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1424/2823] 10.222.0.89 () {58 vars in 1233 bytes} [Thu Jul 28 17:56:25 2022] DELETE /identity/v3/projects/b99edabeefb14f59ade2e81d2670a121 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:56:25.337694 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c29376-aaba-4ea1-8572-cc5ca7585f93 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:25.338193 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c29376-aaba-4ea1-8572-cc5ca7585f93 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:25.338310 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c29376-aaba-4ea1-8572-cc5ca7585f93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:25.358187 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-63c29376-aaba-4ea1-8572-cc5ca7585f93 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:56:25.359400 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-63c29376-aaba-4ea1-8572-cc5ca7585f93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:56:25.418109 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1400/2824] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:56:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:56:25.426165 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2706dc28-5bbd-425f-9b27-7204fc8cbb6f None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:25.426449 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2706dc28-5bbd-425f-9b27-7204fc8cbb6f None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:25.426723 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2706dc28-5bbd-425f-9b27-7204fc8cbb6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:25.446663 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-2706dc28-5bbd-425f-9b27-7204fc8cbb6f None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:56:25.447869 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-2706dc28-5bbd-425f-9b27-7204fc8cbb6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:56:25.455847 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1425/2825] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:56:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:56:25.460709 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c29376-aaba-4ea1-8572-cc5ca7585f93 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:25.470379 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c29376-aaba-4ea1-8572-cc5ca7585f93 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:25.475273 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c102a961-b192-4a59-9658-26be2046f6c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:25.475873 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c102a961-b192-4a59-9658-26be2046f6c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:25.476150 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c102a961-b192-4a59-9658-26be2046f6c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:25.476422 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c102a961-b192-4a59-9658-26be2046f6c8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:25.477112 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c102a961-b192-4a59-9658-26be2046f6c8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80325) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:56:25.477446 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c102a961-b192-4a59-9658-26be2046f6c8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:25.479025 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c102a961-b192-4a59-9658-26be2046f6c8 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:25.490896 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1401/2826] 10.222.0.89 () {58 vars in 1153 bytes} [Thu Jul 28 17:56:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:56:25.495142 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2706dc28-5bbd-425f-9b27-7204fc8cbb6f None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:25.504397 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2706dc28-5bbd-425f-9b27-7204fc8cbb6f None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:25.510168 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95268fe7-f0ff-4398-a3d8-131052c34182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:25.510769 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95268fe7-f0ff-4398-a3d8-131052c34182 admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:25.511051 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95268fe7-f0ff-4398-a3d8-131052c34182 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:25.511319 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95268fe7-f0ff-4398-a3d8-131052c34182 admin admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:25.511978 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95268fe7-f0ff-4398-a3d8-131052c34182 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:25.513896 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95268fe7-f0ff-4398-a3d8-131052c34182 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:25.541804 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1426/2827] 10.222.0.89 () {60 vars in 1153 bytes} [Thu Jul 28 17:56:25 2022] POST /identity/v3/projects => generated 377 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:56:25.545981 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c102a961-b192-4a59-9658-26be2046f6c8 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:25.556775 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c102a961-b192-4a59-9658-26be2046f6c8 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:25.612874 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1912d82d-031d-4558-8bd8-bda99fa0bd26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:25.613508 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1912d82d-031d-4558-8bd8-bda99fa0bd26 admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:25.613790 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1912d82d-031d-4558-8bd8-bda99fa0bd26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:25.614070 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1912d82d-031d-4558-8bd8-bda99fa0bd26 admin admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:25.614792 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1912d82d-031d-4558-8bd8-bda99fa0bd26 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:25.616707 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1912d82d-031d-4558-8bd8-bda99fa0bd26 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:25.638960 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1402/2828] 10.222.0.89 () {60 vars in 1144 bytes} [Thu Jul 28 17:56:25 2022] POST /identity/v3/users => generated 289 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:56:25.643205 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95268fe7-f0ff-4398-a3d8-131052c34182 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:25.654748 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95268fe7-f0ff-4398-a3d8-131052c34182 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:25.659459 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47934b77-40ac-4cb0-9de3-014083b7e73b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:25.660060 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47934b77-40ac-4cb0-9de3-014083b7e73b admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:25.660347 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47934b77-40ac-4cb0-9de3-014083b7e73b admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:25.660621 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47934b77-40ac-4cb0-9de3-014083b7e73b admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:25.661412 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47934b77-40ac-4cb0-9de3-014083b7e73b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:25.663549 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47934b77-40ac-4cb0-9de3-014083b7e73b admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:25.673484 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1427/2829] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:56:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:56:25.677530 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1912d82d-031d-4558-8bd8-bda99fa0bd26 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:25.687477 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1912d82d-031d-4558-8bd8-bda99fa0bd26 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:25.692489 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db6b956-2c66-4fda-9047-ae8f460619a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:25.693727 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db6b956-2c66-4fda-9047-ae8f460619a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:25.694008 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db6b956-2c66-4fda-9047-ae8f460619a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:25.694280 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db6b956-2c66-4fda-9047-ae8f460619a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:25.694978 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db6b956-2c66-4fda-9047-ae8f460619a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:25.696573 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db6b956-2c66-4fda-9047-ae8f460619a0 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:25.707164 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1403/2830] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:56:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:56:25.712063 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47934b77-40ac-4cb0-9de3-014083b7e73b admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:25.722995 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47934b77-40ac-4cb0-9de3-014083b7e73b admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:25.728263 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f8309b-0af2-4cb5-b931-f04bf95ea277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:25.728912 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f8309b-0af2-4cb5-b931-f04bf95ea277 admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:25.729221 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f8309b-0af2-4cb5-b931-f04bf95ea277 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:25.729496 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f8309b-0af2-4cb5-b931-f04bf95ea277 admin admin] PATH_INFO: `/v3/projects/68cbf765c3bc4637aa2f43ae2a7a2db9/users/e4a905bf342346b6b2d44035cddfc1ef/roles/11f8731756fe4caca13efa61b71b11c5` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:25.743956 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f8309b-0af2-4cb5-b931-f04bf95ea277 admin admin] RBAC: Authorizing `identity:create_grant(project_id=68cbf765c3bc4637aa2f43ae2a7a2db9, user_id=e4a905bf342346b6b2d44035cddfc1ef, role_id=11f8731756fe4caca13efa61b71b11c5)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:25.746607 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f8309b-0af2-4cb5-b931-f04bf95ea277 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:25.754598 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:56:25.761876 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1428/2831] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:56:25 2022] PUT /identity/v3/projects/68cbf765c3bc4637aa2f43ae2a7a2db9/users/e4a905bf342346b6b2d44035cddfc1ef/roles/11f8731756fe4caca13efa61b71b11c5 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:56:25.774055 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607f0fca-69d8-4a4f-a75a-835b08afe17a None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:25.774349 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607f0fca-69d8-4a4f-a75a-835b08afe17a None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:25.774628 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607f0fca-69d8-4a4f-a75a-835b08afe17a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:25.803340 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-607f0fca-69d8-4a4f-a75a-835b08afe17a None None] MFA Rules not processed for user `e4a905bf342346b6b2d44035cddfc1ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:56:25.804302 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-607f0fca-69d8-4a4f-a75a-835b08afe17a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:56:25.821606 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:56:25.871219 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1404/2832] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:56:25 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:56:25.875850 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f8309b-0af2-4cb5-b931-f04bf95ea277 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:25.941092 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f8309b-0af2-4cb5-b931-f04bf95ea277 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:25.946685 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd88c87c-abbd-43b7-9cce-944528eff435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:25.947289 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd88c87c-abbd-43b7-9cce-944528eff435 admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:25.947567 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd88c87c-abbd-43b7-9cce-944528eff435 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:25.947837 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd88c87c-abbd-43b7-9cce-944528eff435 admin admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:25.948512 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd88c87c-abbd-43b7-9cce-944528eff435 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:25.950414 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd88c87c-abbd-43b7-9cce-944528eff435 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:25.979386 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1429/2833] 10.222.0.89 () {60 vars in 1153 bytes} [Thu Jul 28 17:56:25 2022] POST /identity/v3/projects => generated 377 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:56:25.983464 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607f0fca-69d8-4a4f-a75a-835b08afe17a None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:25.994770 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607f0fca-69d8-4a4f-a75a-835b08afe17a None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:26.051937 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298037df-d2ff-464a-bcc5-85a7a1652760 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:26.053205 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298037df-d2ff-464a-bcc5-85a7a1652760 admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:26.053490 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298037df-d2ff-464a-bcc5-85a7a1652760 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:26.053822 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298037df-d2ff-464a-bcc5-85a7a1652760 admin admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:26.054513 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298037df-d2ff-464a-bcc5-85a7a1652760 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:26.056420 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298037df-d2ff-464a-bcc5-85a7a1652760 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:26.077941 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1405/2834] 10.222.0.89 () {60 vars in 1144 bytes} [Thu Jul 28 17:56:25 2022] POST /identity/v3/users => generated 289 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:56:26.081927 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd88c87c-abbd-43b7-9cce-944528eff435 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:26.092647 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd88c87c-abbd-43b7-9cce-944528eff435 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:26.097226 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788137cb-7af4-4f43-83eb-294f56764623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:26.097821 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788137cb-7af4-4f43-83eb-294f56764623 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:26.098102 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788137cb-7af4-4f43-83eb-294f56764623 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:26.098378 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788137cb-7af4-4f43-83eb-294f56764623 admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:26.099100 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788137cb-7af4-4f43-83eb-294f56764623 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:26.101277 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788137cb-7af4-4f43-83eb-294f56764623 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:26.111821 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1430/2835] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:56:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:56:26.115831 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298037df-d2ff-464a-bcc5-85a7a1652760 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:26.125862 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298037df-d2ff-464a-bcc5-85a7a1652760 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:26.130541 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c295b0-9907-4c80-a544-2073278e265e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:26.131147 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c295b0-9907-4c80-a544-2073278e265e admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:26.131431 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c295b0-9907-4c80-a544-2073278e265e admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:26.131706 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c295b0-9907-4c80-a544-2073278e265e admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:26.132403 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c295b0-9907-4c80-a544-2073278e265e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:26.134046 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c295b0-9907-4c80-a544-2073278e265e admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:26.144267 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1406/2836] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:56:26 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:56:26.149501 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788137cb-7af4-4f43-83eb-294f56764623 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:26.159931 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788137cb-7af4-4f43-83eb-294f56764623 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:26.165183 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ded9447-cad5-4072-a8ce-2db9621e19c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:26.165843 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ded9447-cad5-4072-a8ce-2db9621e19c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:26.166121 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ded9447-cad5-4072-a8ce-2db9621e19c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:26.166390 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ded9447-cad5-4072-a8ce-2db9621e19c6 admin admin] PATH_INFO: `/v3/projects/216ec75298c942cbbde533ae8807e5cb/users/a8180c2a57574fb88eed48ba53374d8e/roles/11f8731756fe4caca13efa61b71b11c5` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:26.180933 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ded9447-cad5-4072-a8ce-2db9621e19c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=216ec75298c942cbbde533ae8807e5cb, user_id=a8180c2a57574fb88eed48ba53374d8e, role_id=11f8731756fe4caca13efa61b71b11c5)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:26.183621 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ded9447-cad5-4072-a8ce-2db9621e19c6 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:26.197537 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1431/2837] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:56:26 2022] PUT /identity/v3/projects/216ec75298c942cbbde533ae8807e5cb/users/a8180c2a57574fb88eed48ba53374d8e/roles/11f8731756fe4caca13efa61b71b11c5 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:56:26.211092 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f14ac5e-92c0-4a86-ba73-632ba96de657 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:26.211569 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f14ac5e-92c0-4a86-ba73-632ba96de657 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:26.212042 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f14ac5e-92c0-4a86-ba73-632ba96de657 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:26.247105 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-9f14ac5e-92c0-4a86-ba73-632ba96de657 None None] MFA Rules not processed for user `a8180c2a57574fb88eed48ba53374d8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:56:26.248343 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-9f14ac5e-92c0-4a86-ba73-632ba96de657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:56:26.312377 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1407/2838] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:56:26 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:56:26.316868 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ded9447-cad5-4072-a8ce-2db9621e19c6 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:26.378800 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ded9447-cad5-4072-a8ce-2db9621e19c6 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:26.384000 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ac78ba-c4e4-48cb-9bb9-469aee0d453c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:26.384675 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ac78ba-c4e4-48cb-9bb9-469aee0d453c admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:26.384947 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ac78ba-c4e4-48cb-9bb9-469aee0d453c admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:26.385251 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ac78ba-c4e4-48cb-9bb9-469aee0d453c admin admin] PATH_INFO: `/v3/projects` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:26.385925 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ac78ba-c4e4-48cb-9bb9-469aee0d453c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:26.387887 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ac78ba-c4e4-48cb-9bb9-469aee0d453c admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:26.415442 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1432/2839] 10.222.0.89 () {60 vars in 1153 bytes} [Thu Jul 28 17:56:26 2022] POST /identity/v3/projects => generated 373 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:56:26.420275 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f14ac5e-92c0-4a86-ba73-632ba96de657 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:26.431120 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f14ac5e-92c0-4a86-ba73-632ba96de657 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:26.455802 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:56:26.489321 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fa5ab7-bf69-4834-9263-09e89e02cafb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:26.490040 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fa5ab7-bf69-4834-9263-09e89e02cafb admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:26.490413 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fa5ab7-bf69-4834-9263-09e89e02cafb admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:26.490690 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fa5ab7-bf69-4834-9263-09e89e02cafb admin admin] PATH_INFO: `/v3/users` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:26.491373 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fa5ab7-bf69-4834-9263-09e89e02cafb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:26.493374 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fa5ab7-bf69-4834-9263-09e89e02cafb admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:26.522823 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1408/2840] 10.222.0.89 () {60 vars in 1144 bytes} [Thu Jul 28 17:56:26 2022] POST /identity/v3/users => generated 293 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:56:26.527238 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ac78ba-c4e4-48cb-9bb9-469aee0d453c admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:26.537877 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ac78ba-c4e4-48cb-9bb9-469aee0d453c admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:26.542587 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714abe03-2ecd-43bf-a8e2-22819fc6199f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:26.543188 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714abe03-2ecd-43bf-a8e2-22819fc6199f admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:26.543467 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714abe03-2ecd-43bf-a8e2-22819fc6199f admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:26.543739 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714abe03-2ecd-43bf-a8e2-22819fc6199f admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:26.544420 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714abe03-2ecd-43bf-a8e2-22819fc6199f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:26.546033 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714abe03-2ecd-43bf-a8e2-22819fc6199f admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:26.557075 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1433/2841] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:56:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:56:26.561620 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fa5ab7-bf69-4834-9263-09e89e02cafb admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:26.573347 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fa5ab7-bf69-4834-9263-09e89e02cafb admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:26.578331 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc573e2-51ef-49ad-bab3-05e4c48bbdc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:26.579017 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc573e2-51ef-49ad-bab3-05e4c48bbdc8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:26.579433 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc573e2-51ef-49ad-bab3-05e4c48bbdc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:26.579831 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc573e2-51ef-49ad-bab3-05e4c48bbdc8 admin admin] PATH_INFO: `/v3/projects/cf5e31f7a078425e8f35a14f0cabafa2/users/1c31fed4e0f44e3aa33d544828bcbb39/roles/73e457231a824ea98c28d22610be640c` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:26.594390 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc573e2-51ef-49ad-bab3-05e4c48bbdc8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf5e31f7a078425e8f35a14f0cabafa2, user_id=1c31fed4e0f44e3aa33d544828bcbb39, role_id=73e457231a824ea98c28d22610be640c)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:26.597029 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc573e2-51ef-49ad-bab3-05e4c48bbdc8 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:26.610044 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1409/2842] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:56:26 2022] PUT /identity/v3/projects/cf5e31f7a078425e8f35a14f0cabafa2/users/1c31fed4e0f44e3aa33d544828bcbb39/roles/73e457231a824ea98c28d22610be640c => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:56:26.614102 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714abe03-2ecd-43bf-a8e2-22819fc6199f admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:26.625029 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714abe03-2ecd-43bf-a8e2-22819fc6199f admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:26.643752 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:56:26.684040 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c097e762-b5b1-4e63-acd1-60676ca1c9ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:26.684640 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c097e762-b5b1-4e63-acd1-60676ca1c9ee admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:26.684917 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c097e762-b5b1-4e63-acd1-60676ca1c9ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:26.685209 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c097e762-b5b1-4e63-acd1-60676ca1c9ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:26.685888 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c097e762-b5b1-4e63-acd1-60676ca1c9ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:26.687463 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c097e762-b5b1-4e63-acd1-60676ca1c9ee admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:26.697709 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1434/2843] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:56:26 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:56:26.701910 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc573e2-51ef-49ad-bab3-05e4c48bbdc8 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:26.712742 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc573e2-51ef-49ad-bab3-05e4c48bbdc8 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:26.717589 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f964804-5b86-43f6-a474-58de5e78a408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:26.718248 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f964804-5b86-43f6-a474-58de5e78a408 admin admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:26.718531 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f964804-5b86-43f6-a474-58de5e78a408 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:26.718808 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f964804-5b86-43f6-a474-58de5e78a408 admin admin] PATH_INFO: `/v3/projects/cf5e31f7a078425e8f35a14f0cabafa2/users/1c31fed4e0f44e3aa33d544828bcbb39/roles/73e457231a824ea98c28d22610be640c` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:26.720776 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f964804-5b86-43f6-a474-58de5e78a408 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf5e31f7a078425e8f35a14f0cabafa2, user_id=1c31fed4e0f44e3aa33d544828bcbb39, role_id=73e457231a824ea98c28d22610be640c)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:26.723514 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f964804-5b86-43f6-a474-58de5e78a408 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:26.734212 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1410/2844] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:56:26 2022] PUT /identity/v3/projects/cf5e31f7a078425e8f35a14f0cabafa2/users/1c31fed4e0f44e3aa33d544828bcbb39/roles/73e457231a824ea98c28d22610be640c => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:56:26.745467 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4eee04-8fac-4ec0-8e3a-3bd820598d8b None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:26.745754 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4eee04-8fac-4ec0-8e3a-3bd820598d8b None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:26.746027 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4eee04-8fac-4ec0-8e3a-3bd820598d8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:26.774001 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-5e4eee04-8fac-4ec0-8e3a-3bd820598d8b None None] MFA Rules not processed for user `1c31fed4e0f44e3aa33d544828bcbb39`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:56:26.775197 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-5e4eee04-8fac-4ec0-8e3a-3bd820598d8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:56:26.844903 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1435/2845] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:56:26 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:56:26.856688 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f964804-5b86-43f6-a474-58de5e78a408 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:26.858515 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-5f964804-5b86-43f6-a474-58de5e78a408 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:56:26.922178 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f964804-5b86-43f6-a474-58de5e78a408 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:26.925982 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5603db-5f20-495e-9c23-873bb998a992 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96d512d0c84240918dae6c4a3e8b7b79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:26.926621 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5603db-5f20-495e-9c23-873bb998a992 service cinder] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:26.926883 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5603db-5f20-495e-9c23-873bb998a992 service cinder] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:26.927634 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5603db-5f20-495e-9c23-873bb998a992 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:26.983728 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5603db-5f20-495e-9c23-873bb998a992 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:26.985178 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5603db-5f20-495e-9c23-873bb998a992 service cinder] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:26.994864 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1411/2846] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:56:26 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:56:28.503564 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4eee04-8fac-4ec0-8e3a-3bd820598d8b None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:28.504981 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-5e4eee04-8fac-4ec0-8e3a-3bd820598d8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:56:28.517823 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4eee04-8fac-4ec0-8e3a-3bd820598d8b None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:28.522407 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664a7bdf-19f8-4fad-9e03-d1efd5611378 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96d512d0c84240918dae6c4a3e8b7b79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:28.523006 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664a7bdf-19f8-4fad-9e03-d1efd5611378 service cinder] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:28.523281 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664a7bdf-19f8-4fad-9e03-d1efd5611378 service cinder] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:28.523547 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664a7bdf-19f8-4fad-9e03-d1efd5611378 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:28.581776 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664a7bdf-19f8-4fad-9e03-d1efd5611378 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:28.583214 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664a7bdf-19f8-4fad-9e03-d1efd5611378 service cinder] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:28.593101 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1436/2847] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:56:28 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:56:29.311939 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5603db-5f20-495e-9c23-873bb998a992 service cinder] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:29.321320 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5603db-5f20-495e-9c23-873bb998a992 service cinder] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:29.324998 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521107fd-b97a-4d32-bfb5-1d05601d85dc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96d512d0c84240918dae6c4a3e8b7b79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:29.325643 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521107fd-b97a-4d32-bfb5-1d05601d85dc service cinder] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:29.325922 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521107fd-b97a-4d32-bfb5-1d05601d85dc service cinder] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:29.326199 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521107fd-b97a-4d32-bfb5-1d05601d85dc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:29.335852 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521107fd-b97a-4d32-bfb5-1d05601d85dc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:29.337296 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521107fd-b97a-4d32-bfb5-1d05601d85dc service cinder] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:29.346587 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1412/2848] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:56:29 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:56:33.235787 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664a7bdf-19f8-4fad-9e03-d1efd5611378 service cinder] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:33.282030 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=100 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:56:33.300871 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664a7bdf-19f8-4fad-9e03-d1efd5611378 service cinder] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:33.306204 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65af95ea-5085-49c1-be3b-c9ff16270279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:33.306832 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65af95ea-5085-49c1-be3b-c9ff16270279 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:33.307113 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65af95ea-5085-49c1-be3b-c9ff16270279 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:33.307388 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65af95ea-5085-49c1-be3b-c9ff16270279 admin admin] PATH_INFO: `/v3/users/e4a905bf342346b6b2d44035cddfc1ef` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:33.308560 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65af95ea-5085-49c1-be3b-c9ff16270279 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4a905bf342346b6b2d44035cddfc1ef)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:33.310516 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65af95ea-5085-49c1-be3b-c9ff16270279 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:33.356618 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-65af95ea-5085-49c1-be3b-c9ff16270279 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4a905bf342346b6b2d44035cddfc1ef'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:56:33.363358 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-65af95ea-5085-49c1-be3b-c9ff16270279 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4a905bf342346b6b2d44035cddfc1ef'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:56:33.364029 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-65af95ea-5085-49c1-be3b-c9ff16270279 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4a905bf342346b6b2d44035cddfc1ef'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:56:33.373075 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-65af95ea-5085-49c1-be3b-c9ff16270279 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4a905bf342346b6b2d44035cddfc1ef'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:56:33.385664 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-65af95ea-5085-49c1-be3b-c9ff16270279 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4a905bf342346b6b2d44035cddfc1ef'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:56:33.391774 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1437/2849] 10.222.0.89 () {58 vars in 1224 bytes} [Thu Jul 28 17:56:33 2022] DELETE /identity/v3/users/e4a905bf342346b6b2d44035cddfc1ef => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:56:33.396458 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4a9784-4ce9-4dea-80e5-8bf2ad64937b None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:33.396951 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4a9784-4ce9-4dea-80e5-8bf2ad64937b None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:33.397057 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4a9784-4ce9-4dea-80e5-8bf2ad64937b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:33.416135 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-3c4a9784-4ce9-4dea-80e5-8bf2ad64937b None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:56:33.417379 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-3c4a9784-4ce9-4dea-80e5-8bf2ad64937b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:56:33.431412 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:56:33.479113 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1413/2850] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:56:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:56:33.490119 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65af95ea-5085-49c1-be3b-c9ff16270279 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:33.491663 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-65af95ea-5085-49c1-be3b-c9ff16270279 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:56:33.553196 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65af95ea-5085-49c1-be3b-c9ff16270279 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:33.556890 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579b7b50-d672-434b-8419-7818e8703a1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76817cd541294331b0023069092e8e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:33.557534 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579b7b50-d672-434b-8419-7818e8703a1c service neutron] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:33.557815 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579b7b50-d672-434b-8419-7818e8703a1c service neutron] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:33.558091 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579b7b50-d672-434b-8419-7818e8703a1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:33.567863 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579b7b50-d672-434b-8419-7818e8703a1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:33.569325 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579b7b50-d672-434b-8419-7818e8703a1c service neutron] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:33.580147 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1438/2851] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:56:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:56:33.865259 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4a9784-4ce9-4dea-80e5-8bf2ad64937b None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:33.866779 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-3c4a9784-4ce9-4dea-80e5-8bf2ad64937b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:56:33.881889 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4a9784-4ce9-4dea-80e5-8bf2ad64937b None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:33.887060 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38db07ea-11ab-4d79-a54b-eaf39bf32c6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:33.887896 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38db07ea-11ab-4d79-a54b-eaf39bf32c6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:33.888215 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38db07ea-11ab-4d79-a54b-eaf39bf32c6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:33.888445 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38db07ea-11ab-4d79-a54b-eaf39bf32c6e admin admin] PATH_INFO: `/v3/projects/68cbf765c3bc4637aa2f43ae2a7a2db9` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:33.889649 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38db07ea-11ab-4d79-a54b-eaf39bf32c6e admin admin] RBAC: Authorizing `identity:delete_project(project_id=68cbf765c3bc4637aa2f43ae2a7a2db9)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:33.891564 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38db07ea-11ab-4d79-a54b-eaf39bf32c6e admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:33.940016 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-38db07ea-11ab-4d79-a54b-eaf39bf32c6e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68cbf765c3bc4637aa2f43ae2a7a2db9', 'request_id': 'req-38db07ea-11ab-4d79-a54b-eaf39bf32c6e'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:56:33.946767 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-38db07ea-11ab-4d79-a54b-eaf39bf32c6e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68cbf765c3bc4637aa2f43ae2a7a2db9', 'request_id': 'req-38db07ea-11ab-4d79-a54b-eaf39bf32c6e'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:56:33.952617 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-38db07ea-11ab-4d79-a54b-eaf39bf32c6e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68cbf765c3bc4637aa2f43ae2a7a2db9', 'request_id': 'req-38db07ea-11ab-4d79-a54b-eaf39bf32c6e'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:56:33.966085 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-38db07ea-11ab-4d79-a54b-eaf39bf32c6e admin admin] The token cache is being invalidate because project 68cbf765c3bc4637aa2f43ae2a7a2db9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80325) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 17:56:33.966377 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-38db07ea-11ab-4d79-a54b-eaf39bf32c6e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:56:33.967007 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-38db07ea-11ab-4d79-a54b-eaf39bf32c6e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:56:33.968526 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1414/2852] 10.222.0.89 () {58 vars in 1233 bytes} [Thu Jul 28 17:56:33 2022] DELETE /identity/v3/projects/68cbf765c3bc4637aa2f43ae2a7a2db9 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:56:33.972895 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579b7b50-d672-434b-8419-7818e8703a1c service neutron] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:33.983642 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579b7b50-d672-434b-8419-7818e8703a1c service neutron] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:34.041172 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdec044-e149-4ab2-9daa-b44d5fc0a842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:34.041824 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdec044-e149-4ab2-9daa-b44d5fc0a842 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:34.042102 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdec044-e149-4ab2-9daa-b44d5fc0a842 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:34.042377 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdec044-e149-4ab2-9daa-b44d5fc0a842 admin admin] PATH_INFO: `/v3/users/a8180c2a57574fb88eed48ba53374d8e` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:34.043544 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdec044-e149-4ab2-9daa-b44d5fc0a842 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8180c2a57574fb88eed48ba53374d8e)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:34.045481 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdec044-e149-4ab2-9daa-b44d5fc0a842 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:34.050038 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:56:34.093098 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-0fdec044-e149-4ab2-9daa-b44d5fc0a842 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8180c2a57574fb88eed48ba53374d8e'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:56:34.100340 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-0fdec044-e149-4ab2-9daa-b44d5fc0a842 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8180c2a57574fb88eed48ba53374d8e'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:56:34.101033 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-0fdec044-e149-4ab2-9daa-b44d5fc0a842 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8180c2a57574fb88eed48ba53374d8e'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:56:34.108679 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-0fdec044-e149-4ab2-9daa-b44d5fc0a842 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8180c2a57574fb88eed48ba53374d8e'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:56:34.120977 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-0fdec044-e149-4ab2-9daa-b44d5fc0a842 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8180c2a57574fb88eed48ba53374d8e'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:56:34.126964 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1439/2853] 10.222.0.89 () {58 vars in 1224 bytes} [Thu Jul 28 17:56:33 2022] DELETE /identity/v3/users/a8180c2a57574fb88eed48ba53374d8e => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:56:34.407118 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38db07ea-11ab-4d79-a54b-eaf39bf32c6e admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:34.408648 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-38db07ea-11ab-4d79-a54b-eaf39bf32c6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:56:34.422460 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38db07ea-11ab-4d79-a54b-eaf39bf32c6e admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:34.480219 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e94a09-0ea2-4df4-bdda-5f9cf922a267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:34.480778 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e94a09-0ea2-4df4-bdda-5f9cf922a267 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:34.481055 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e94a09-0ea2-4df4-bdda-5f9cf922a267 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:34.481357 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e94a09-0ea2-4df4-bdda-5f9cf922a267 admin admin] PATH_INFO: `/v3/projects/216ec75298c942cbbde533ae8807e5cb` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:34.482515 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e94a09-0ea2-4df4-bdda-5f9cf922a267 admin admin] RBAC: Authorizing `identity:delete_project(project_id=216ec75298c942cbbde533ae8807e5cb)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:34.484403 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e94a09-0ea2-4df4-bdda-5f9cf922a267 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:34.528013 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:56:34.531246 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-c2e94a09-0ea2-4df4-bdda-5f9cf922a267 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '216ec75298c942cbbde533ae8807e5cb', 'request_id': 'req-c2e94a09-0ea2-4df4-bdda-5f9cf922a267'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:56:34.539904 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-c2e94a09-0ea2-4df4-bdda-5f9cf922a267 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '216ec75298c942cbbde533ae8807e5cb', 'request_id': 'req-c2e94a09-0ea2-4df4-bdda-5f9cf922a267'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:56:34.545538 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-c2e94a09-0ea2-4df4-bdda-5f9cf922a267 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '216ec75298c942cbbde533ae8807e5cb', 'request_id': 'req-c2e94a09-0ea2-4df4-bdda-5f9cf922a267'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:56:34.558331 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-c2e94a09-0ea2-4df4-bdda-5f9cf922a267 admin admin] The token cache is being invalidate because project 216ec75298c942cbbde533ae8807e5cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80325) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 17:56:34.558623 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-c2e94a09-0ea2-4df4-bdda-5f9cf922a267 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:56:34.559246 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-c2e94a09-0ea2-4df4-bdda-5f9cf922a267 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:56:34.560755 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1415/2854] 10.222.0.89 () {58 vars in 1233 bytes} [Thu Jul 28 17:56:34 2022] DELETE /identity/v3/projects/216ec75298c942cbbde533ae8807e5cb => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:56:34.564743 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdec044-e149-4ab2-9daa-b44d5fc0a842 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:34.566225 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-0fdec044-e149-4ab2-9daa-b44d5fc0a842 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:56:34.579508 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdec044-e149-4ab2-9daa-b44d5fc0a842 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:34.636620 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee84781f-2248-4488-aa7e-ab3bb84126ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:34.637332 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee84781f-2248-4488-aa7e-ab3bb84126ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:34.637638 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee84781f-2248-4488-aa7e-ab3bb84126ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:34.637917 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee84781f-2248-4488-aa7e-ab3bb84126ba admin admin] PATH_INFO: `/v3/users/1c31fed4e0f44e3aa33d544828bcbb39` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:34.639120 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee84781f-2248-4488-aa7e-ab3bb84126ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c31fed4e0f44e3aa33d544828bcbb39)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:34.641183 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee84781f-2248-4488-aa7e-ab3bb84126ba admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:34.688665 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-ee84781f-2248-4488-aa7e-ab3bb84126ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c31fed4e0f44e3aa33d544828bcbb39'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:56:34.695804 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-ee84781f-2248-4488-aa7e-ab3bb84126ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c31fed4e0f44e3aa33d544828bcbb39'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:56:34.696479 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-ee84781f-2248-4488-aa7e-ab3bb84126ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c31fed4e0f44e3aa33d544828bcbb39'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:56:34.703882 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-ee84781f-2248-4488-aa7e-ab3bb84126ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c31fed4e0f44e3aa33d544828bcbb39'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:56:34.715740 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-ee84781f-2248-4488-aa7e-ab3bb84126ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c31fed4e0f44e3aa33d544828bcbb39'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:56:34.721515 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1440/2855] 10.222.0.89 () {58 vars in 1224 bytes} [Thu Jul 28 17:56:34 2022] DELETE /identity/v3/users/1c31fed4e0f44e3aa33d544828bcbb39 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:56:35.014795 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e94a09-0ea2-4df4-bdda-5f9cf922a267 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:35.016246 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-c2e94a09-0ea2-4df4-bdda-5f9cf922a267 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:56:35.030005 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e94a09-0ea2-4df4-bdda-5f9cf922a267 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:35.088808 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb2dd4f-cdbf-400c-9646-bb6c10244286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:35.089485 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb2dd4f-cdbf-400c-9646-bb6c10244286 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:35.089773 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb2dd4f-cdbf-400c-9646-bb6c10244286 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:35.090049 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb2dd4f-cdbf-400c-9646-bb6c10244286 admin admin] PATH_INFO: `/v3/projects/cf5e31f7a078425e8f35a14f0cabafa2` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:35.091216 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb2dd4f-cdbf-400c-9646-bb6c10244286 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf5e31f7a078425e8f35a14f0cabafa2)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:35.093342 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb2dd4f-cdbf-400c-9646-bb6c10244286 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:35.139881 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-4bb2dd4f-cdbf-400c-9646-bb6c10244286 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf5e31f7a078425e8f35a14f0cabafa2', 'request_id': 'req-4bb2dd4f-cdbf-400c-9646-bb6c10244286'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:56:35.146419 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-4bb2dd4f-cdbf-400c-9646-bb6c10244286 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf5e31f7a078425e8f35a14f0cabafa2', 'request_id': 'req-4bb2dd4f-cdbf-400c-9646-bb6c10244286'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:56:35.152212 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-4bb2dd4f-cdbf-400c-9646-bb6c10244286 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf5e31f7a078425e8f35a14f0cabafa2', 'request_id': 'req-4bb2dd4f-cdbf-400c-9646-bb6c10244286'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:56:35.165107 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-4bb2dd4f-cdbf-400c-9646-bb6c10244286 admin admin] The token cache is being invalidate because project cf5e31f7a078425e8f35a14f0cabafa2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80325) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 17:56:35.165438 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-4bb2dd4f-cdbf-400c-9646-bb6c10244286 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:56:35.166070 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-4bb2dd4f-cdbf-400c-9646-bb6c10244286 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:56:35.167741 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1416/2856] 10.222.0.89 () {58 vars in 1233 bytes} [Thu Jul 28 17:56:35 2022] DELETE /identity/v3/projects/cf5e31f7a078425e8f35a14f0cabafa2 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:56:35.172605 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392e18b9-def8-4036-8c61-1d1857811930 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:35.172891 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392e18b9-def8-4036-8c61-1d1857811930 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:35.173177 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392e18b9-def8-4036-8c61-1d1857811930 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:35.180908 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:56:35.195054 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-392e18b9-def8-4036-8c61-1d1857811930 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:56:35.196293 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-392e18b9-def8-4036-8c61-1d1857811930 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:56:35.261330 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1441/2857] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:56:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:56:35.269233 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e87617-136a-4d19-8b81-cb52e75c8335 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:35.269518 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e87617-136a-4d19-8b81-cb52e75c8335 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:35.269788 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e87617-136a-4d19-8b81-cb52e75c8335 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:35.293031 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-41e87617-136a-4d19-8b81-cb52e75c8335 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:56:35.294771 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-41e87617-136a-4d19-8b81-cb52e75c8335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:56:35.303787 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1417/2858] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:56:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:56:35.308346 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392e18b9-def8-4036-8c61-1d1857811930 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:35.318861 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392e18b9-def8-4036-8c61-1d1857811930 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:35.323601 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf44c44a-1ac6-415a-b806-6aef357c97d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:35.324205 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf44c44a-1ac6-415a-b806-6aef357c97d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:35.324485 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf44c44a-1ac6-415a-b806-6aef357c97d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:35.324756 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf44c44a-1ac6-415a-b806-6aef357c97d3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:35.325482 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf44c44a-1ac6-415a-b806-6aef357c97d3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:56:35.325787 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf44c44a-1ac6-415a-b806-6aef357c97d3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:35.327363 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf44c44a-1ac6-415a-b806-6aef357c97d3 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:35.340336 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1442/2859] 10.222.0.89 () {58 vars in 1153 bytes} [Thu Jul 28 17:56:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:56:35.345063 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e87617-136a-4d19-8b81-cb52e75c8335 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:35.354795 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e87617-136a-4d19-8b81-cb52e75c8335 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:35.359771 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6313d1-c28a-4dec-bf18-d74fbbcf41a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:35.360377 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6313d1-c28a-4dec-bf18-d74fbbcf41a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:35.360657 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6313d1-c28a-4dec-bf18-d74fbbcf41a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:35.360932 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6313d1-c28a-4dec-bf18-d74fbbcf41a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:35.361686 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6313d1-c28a-4dec-bf18-d74fbbcf41a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:35.363757 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6313d1-c28a-4dec-bf18-d74fbbcf41a4 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:35.391142 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1418/2860] 10.222.0.89 () {60 vars in 1153 bytes} [Thu Jul 28 17:56:35 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:56:35.395531 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf44c44a-1ac6-415a-b806-6aef357c97d3 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:35.406546 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf44c44a-1ac6-415a-b806-6aef357c97d3 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:35.462783 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9c0432-f412-44dd-b2c2-0efec5cc511d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:35.463387 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9c0432-f412-44dd-b2c2-0efec5cc511d admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:35.463664 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9c0432-f412-44dd-b2c2-0efec5cc511d admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:35.463953 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9c0432-f412-44dd-b2c2-0efec5cc511d admin admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:35.464619 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9c0432-f412-44dd-b2c2-0efec5cc511d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:35.466543 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9c0432-f412-44dd-b2c2-0efec5cc511d admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:35.487806 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1443/2861] 10.222.0.89 () {60 vars in 1144 bytes} [Thu Jul 28 17:56:35 2022] POST /identity/v3/users => generated 293 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:56:35.492499 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6313d1-c28a-4dec-bf18-d74fbbcf41a4 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:35.502917 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6313d1-c28a-4dec-bf18-d74fbbcf41a4 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:35.508259 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a535735-5650-470c-87ed-28269a0f47c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:35.508865 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a535735-5650-470c-87ed-28269a0f47c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:35.509148 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a535735-5650-470c-87ed-28269a0f47c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:35.509453 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a535735-5650-470c-87ed-28269a0f47c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:35.510177 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a535735-5650-470c-87ed-28269a0f47c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:35.511861 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a535735-5650-470c-87ed-28269a0f47c2 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:35.522747 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1419/2862] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:56:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:56:35.526995 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9c0432-f412-44dd-b2c2-0efec5cc511d admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:35.537527 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9c0432-f412-44dd-b2c2-0efec5cc511d admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:35.542133 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68885714-0342-4f5f-b653-02fd5ba4d94e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:35.542736 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68885714-0342-4f5f-b653-02fd5ba4d94e admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:35.543056 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68885714-0342-4f5f-b653-02fd5ba4d94e admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:35.543298 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68885714-0342-4f5f-b653-02fd5ba4d94e admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:35.543981 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68885714-0342-4f5f-b653-02fd5ba4d94e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:35.545597 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68885714-0342-4f5f-b653-02fd5ba4d94e admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:35.556980 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1444/2863] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:56:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 17:56:35.561342 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a535735-5650-470c-87ed-28269a0f47c2 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:35.572391 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a535735-5650-470c-87ed-28269a0f47c2 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:35.577320 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ce3982-f27e-429c-810d-b88a7f43317e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:35.578002 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ce3982-f27e-429c-810d-b88a7f43317e admin admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:35.578304 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ce3982-f27e-429c-810d-b88a7f43317e admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:35.578630 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ce3982-f27e-429c-810d-b88a7f43317e admin admin] PATH_INFO: `/v3/projects/e998a20910434d64a2cd61825080927a/users/24b7853ae0ee460694c2a8ac940868aa/roles/11f8731756fe4caca13efa61b71b11c5` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:35.594067 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ce3982-f27e-429c-810d-b88a7f43317e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e998a20910434d64a2cd61825080927a, user_id=24b7853ae0ee460694c2a8ac940868aa, role_id=11f8731756fe4caca13efa61b71b11c5)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:35.596969 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ce3982-f27e-429c-810d-b88a7f43317e admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:35.606976 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:56:35.613637 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1420/2864] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:56:35 2022] PUT /identity/v3/projects/e998a20910434d64a2cd61825080927a/users/24b7853ae0ee460694c2a8ac940868aa/roles/11f8731756fe4caca13efa61b71b11c5 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:56:35.624828 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8859c12-a475-45a7-bdcb-cd32da549b5a None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:35.625478 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8859c12-a475-45a7-bdcb-cd32da549b5a None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:35.625478 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8859c12-a475-45a7-bdcb-cd32da549b5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:35.655165 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-d8859c12-a475-45a7-bdcb-cd32da549b5a None None] MFA Rules not processed for user `24b7853ae0ee460694c2a8ac940868aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:56:35.656379 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-d8859c12-a475-45a7-bdcb-cd32da549b5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:56:35.713773 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:56:35.719670 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1445/2865] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:56:35 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:56:35.731433 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ce3982-f27e-429c-810d-b88a7f43317e admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:35.733399 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-31ce3982-f27e-429c-810d-b88a7f43317e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:56:35.792002 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ce3982-f27e-429c-810d-b88a7f43317e admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:35.796141 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5ecf26-a278-404b-b9c6-ae98b7389eb0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96d512d0c84240918dae6c4a3e8b7b79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:35.796746 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5ecf26-a278-404b-b9c6-ae98b7389eb0 service cinder] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:35.797021 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5ecf26-a278-404b-b9c6-ae98b7389eb0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:35.797318 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5ecf26-a278-404b-b9c6-ae98b7389eb0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:35.807107 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5ecf26-a278-404b-b9c6-ae98b7389eb0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:35.808548 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5ecf26-a278-404b-b9c6-ae98b7389eb0 service cinder] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:35.818400 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1421/2866] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:56:35 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:56:38.403473 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8859c12-a475-45a7-bdcb-cd32da549b5a None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:38.465450 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8859c12-a475-45a7-bdcb-cd32da549b5a None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:38.469962 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cce151-3e07-4e1f-8609-92a1a253916b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:38.470587 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cce151-3e07-4e1f-8609-92a1a253916b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:38.470863 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cce151-3e07-4e1f-8609-92a1a253916b admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:38.471133 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cce151-3e07-4e1f-8609-92a1a253916b admin admin] PATH_INFO: `/v3/users/24b7853ae0ee460694c2a8ac940868aa` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:38.472280 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cce151-3e07-4e1f-8609-92a1a253916b admin admin] RBAC: Authorizing `identity:delete_user(user_id=24b7853ae0ee460694c2a8ac940868aa)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:38.474821 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cce151-3e07-4e1f-8609-92a1a253916b admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:38.519989 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-91cce151-3e07-4e1f-8609-92a1a253916b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24b7853ae0ee460694c2a8ac940868aa'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:56:38.527328 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-91cce151-3e07-4e1f-8609-92a1a253916b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24b7853ae0ee460694c2a8ac940868aa'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:56:38.528051 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-91cce151-3e07-4e1f-8609-92a1a253916b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24b7853ae0ee460694c2a8ac940868aa'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:56:38.535082 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-91cce151-3e07-4e1f-8609-92a1a253916b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24b7853ae0ee460694c2a8ac940868aa'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:56:38.547837 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-91cce151-3e07-4e1f-8609-92a1a253916b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24b7853ae0ee460694c2a8ac940868aa'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:56:38.555001 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1446/2867] 10.222.0.89 () {58 vars in 1224 bytes} [Thu Jul 28 17:56:38 2022] DELETE /identity/v3/users/24b7853ae0ee460694c2a8ac940868aa => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:56:38.559599 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777611d6-b730-4808-a0e2-8d727d5af80b None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:38.560090 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777611d6-b730-4808-a0e2-8d727d5af80b None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:38.560200 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777611d6-b730-4808-a0e2-8d727d5af80b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:38.578781 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-777611d6-b730-4808-a0e2-8d727d5af80b None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:56:38.579996 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-777611d6-b730-4808-a0e2-8d727d5af80b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:56:38.637136 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1422/2868] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:56:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:56:38.646841 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cce151-3e07-4e1f-8609-92a1a253916b admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:38.648341 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-91cce151-3e07-4e1f-8609-92a1a253916b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:56:38.708506 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cce151-3e07-4e1f-8609-92a1a253916b admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:38.712432 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c4c8e6-5f61-478c-83a6-8e00f95bb20a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76817cd541294331b0023069092e8e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:38.713052 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c4c8e6-5f61-478c-83a6-8e00f95bb20a service neutron] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:38.713414 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c4c8e6-5f61-478c-83a6-8e00f95bb20a service neutron] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:38.713687 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c4c8e6-5f61-478c-83a6-8e00f95bb20a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:38.722670 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c4c8e6-5f61-478c-83a6-8e00f95bb20a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:38.724013 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c4c8e6-5f61-478c-83a6-8e00f95bb20a service neutron] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:38.734540 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1447/2869] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:56:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:56:39.000048 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777611d6-b730-4808-a0e2-8d727d5af80b None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:39.001592 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-777611d6-b730-4808-a0e2-8d727d5af80b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:56:39.017012 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777611d6-b730-4808-a0e2-8d727d5af80b None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:39.022487 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30894f57-09e2-4cc2-ae80-4653e21a2952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:39.023683 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30894f57-09e2-4cc2-ae80-4653e21a2952 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:39.023990 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30894f57-09e2-4cc2-ae80-4653e21a2952 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:39.024442 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30894f57-09e2-4cc2-ae80-4653e21a2952 admin admin] PATH_INFO: `/v3/projects/e998a20910434d64a2cd61825080927a` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:39.025988 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30894f57-09e2-4cc2-ae80-4653e21a2952 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e998a20910434d64a2cd61825080927a)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:39.027909 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30894f57-09e2-4cc2-ae80-4653e21a2952 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:39.081227 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-30894f57-09e2-4cc2-ae80-4653e21a2952 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e998a20910434d64a2cd61825080927a', 'request_id': 'req-30894f57-09e2-4cc2-ae80-4653e21a2952'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:56:39.088008 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-30894f57-09e2-4cc2-ae80-4653e21a2952 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e998a20910434d64a2cd61825080927a', 'request_id': 'req-30894f57-09e2-4cc2-ae80-4653e21a2952'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:56:39.093709 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-30894f57-09e2-4cc2-ae80-4653e21a2952 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e998a20910434d64a2cd61825080927a', 'request_id': 'req-30894f57-09e2-4cc2-ae80-4653e21a2952'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:56:39.108306 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-30894f57-09e2-4cc2-ae80-4653e21a2952 admin admin] The token cache is being invalidate because project e998a20910434d64a2cd61825080927a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80325) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 17:56:39.108618 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-30894f57-09e2-4cc2-ae80-4653e21a2952 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:56:39.109256 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-30894f57-09e2-4cc2-ae80-4653e21a2952 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:56:39.110787 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1423/2870] 10.222.0.89 () {58 vars in 1233 bytes} [Thu Jul 28 17:56:38 2022] DELETE /identity/v3/projects/e998a20910434d64a2cd61825080927a => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:56:39.116231 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93ab3a5-0c6e-49ae-95f5-ca8387988c19 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:39.116520 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93ab3a5-0c6e-49ae-95f5-ca8387988c19 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:39.116800 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93ab3a5-0c6e-49ae-95f5-ca8387988c19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:39.136531 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-e93ab3a5-0c6e-49ae-95f5-ca8387988c19 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:56:39.137784 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-e93ab3a5-0c6e-49ae-95f5-ca8387988c19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:56:39.151793 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:56:39.198896 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1448/2871] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:56:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:56:39.208519 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56e0f76-ada6-4192-9d25-1d04e736c28a None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:39.208804 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56e0f76-ada6-4192-9d25-1d04e736c28a None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:39.209963 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56e0f76-ada6-4192-9d25-1d04e736c28a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:39.230306 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-c56e0f76-ada6-4192-9d25-1d04e736c28a None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:56:39.231546 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-c56e0f76-ada6-4192-9d25-1d04e736c28a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:56:39.239556 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1424/2872] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:56:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:56:39.244518 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93ab3a5-0c6e-49ae-95f5-ca8387988c19 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:39.254107 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93ab3a5-0c6e-49ae-95f5-ca8387988c19 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:39.259425 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed36b60-33db-4049-b9c6-138cae67c8c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:39.259948 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed36b60-33db-4049-b9c6-138cae67c8c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:39.260226 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed36b60-33db-4049-b9c6-138cae67c8c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:39.260500 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed36b60-33db-4049-b9c6-138cae67c8c9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:39.261215 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed36b60-33db-4049-b9c6-138cae67c8c9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80324) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 17:56:39.261524 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed36b60-33db-4049-b9c6-138cae67c8c9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:39.263102 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed36b60-33db-4049-b9c6-138cae67c8c9 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:39.275174 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1449/2873] 10.222.0.89 () {58 vars in 1153 bytes} [Thu Jul 28 17:56:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 17:56:39.280004 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56e0f76-ada6-4192-9d25-1d04e736c28a None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:39.291330 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56e0f76-ada6-4192-9d25-1d04e736c28a None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:39.297437 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12798f1-8483-419b-a21e-2ea508f8b140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:39.298061 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12798f1-8483-419b-a21e-2ea508f8b140 admin admin] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:39.298342 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12798f1-8483-419b-a21e-2ea508f8b140 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:39.298617 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12798f1-8483-419b-a21e-2ea508f8b140 admin admin] PATH_INFO: `/v3/projects` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:39.299306 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12798f1-8483-419b-a21e-2ea508f8b140 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:39.301237 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12798f1-8483-419b-a21e-2ea508f8b140 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:39.323677 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:56:39.335576 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1425/2874] 10.222.0.89 () {60 vars in 1153 bytes} [Thu Jul 28 17:56:39 2022] POST /identity/v3/projects => generated 381 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:56:39.339627 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed36b60-33db-4049-b9c6-138cae67c8c9 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:39.350931 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed36b60-33db-4049-b9c6-138cae67c8c9 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:39.411452 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264ff014-b929-42f0-af22-1ef446ed7f7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:39.412699 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264ff014-b929-42f0-af22-1ef446ed7f7e admin admin] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:39.412935 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264ff014-b929-42f0-af22-1ef446ed7f7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:39.412935 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264ff014-b929-42f0-af22-1ef446ed7f7e admin admin] PATH_INFO: `/v3/users` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:39.413811 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264ff014-b929-42f0-af22-1ef446ed7f7e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:39.415719 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264ff014-b929-42f0-af22-1ef446ed7f7e admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:39.441470 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1450/2875] 10.222.0.89 () {60 vars in 1144 bytes} [Thu Jul 28 17:56:39 2022] POST /identity/v3/users => generated 291 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 17:56:39.445866 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12798f1-8483-419b-a21e-2ea508f8b140 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:39.456359 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12798f1-8483-419b-a21e-2ea508f8b140 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:39.461091 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0d74d3-f215-41e8-9040-e428a205020b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:39.461739 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0d74d3-f215-41e8-9040-e428a205020b admin admin] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:39.462018 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0d74d3-f215-41e8-9040-e428a205020b admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:39.462294 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0d74d3-f215-41e8-9040-e428a205020b admin admin] PATH_INFO: `/v3/roles` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:39.462991 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0d74d3-f215-41e8-9040-e428a205020b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:39.464580 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0d74d3-f215-41e8-9040-e428a205020b admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:39.474440 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1426/2876] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:56:39 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:56:39.479039 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264ff014-b929-42f0-af22-1ef446ed7f7e admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:39.489119 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264ff014-b929-42f0-af22-1ef446ed7f7e admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:39.494693 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7fe1d7-b58f-49b7-9c76-d25067cb54b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:39.495238 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7fe1d7-b58f-49b7-9c76-d25067cb54b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:39.495552 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7fe1d7-b58f-49b7-9c76-d25067cb54b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:39.495830 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7fe1d7-b58f-49b7-9c76-d25067cb54b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:39.496523 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7fe1d7-b58f-49b7-9c76-d25067cb54b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:39.498145 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7fe1d7-b58f-49b7-9c76-d25067cb54b6 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:39.508528 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1451/2877] 10.222.0.89 () {58 vars in 1122 bytes} [Thu Jul 28 17:56:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 17:56:39.512573 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0d74d3-f215-41e8-9040-e428a205020b admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:39.523024 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0d74d3-f215-41e8-9040-e428a205020b admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:39.527946 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32f9149-e4d1-4957-b8f3-90f26a6fe6b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:39.528600 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32f9149-e4d1-4957-b8f3-90f26a6fe6b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:39.528878 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32f9149-e4d1-4957-b8f3-90f26a6fe6b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:39.529152 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32f9149-e4d1-4957-b8f3-90f26a6fe6b4 admin admin] PATH_INFO: `/v3/projects/6ed3b5ffc1dd4fb9a814a967d1c9e8dc/users/2277cf311617495cb784285f328451c5/roles/11f8731756fe4caca13efa61b71b11c5` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:39.544897 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32f9149-e4d1-4957-b8f3-90f26a6fe6b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ed3b5ffc1dd4fb9a814a967d1c9e8dc, user_id=2277cf311617495cb784285f328451c5, role_id=11f8731756fe4caca13efa61b71b11c5)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:39.547593 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32f9149-e4d1-4957-b8f3-90f26a6fe6b4 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:39.560857 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1427/2878] 10.222.0.89 () {60 vars in 1483 bytes} [Thu Jul 28 17:56:39 2022] PUT /identity/v3/projects/6ed3b5ffc1dd4fb9a814a967d1c9e8dc/users/2277cf311617495cb784285f328451c5/roles/11f8731756fe4caca13efa61b71b11c5 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:56:39.565839 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad183d15-87ee-4de6-b421-1ac45730a836 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:39.566152 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad183d15-87ee-4de6-b421-1ac45730a836 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:39.566428 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad183d15-87ee-4de6-b421-1ac45730a836 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:39.585919 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-ad183d15-87ee-4de6-b421-1ac45730a836 None None] MFA Rules not processed for user `bbc7ab320041413396df1eef4db728ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:56:39.587132 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-ad183d15-87ee-4de6-b421-1ac45730a836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:56:39.644128 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1452/2879] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:56:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:56:39.655213 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32f9149-e4d1-4957-b8f3-90f26a6fe6b4 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:39.657110 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-d32f9149-e4d1-4957-b8f3-90f26a6fe6b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:56:39.714008 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32f9149-e4d1-4957-b8f3-90f26a6fe6b4 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:39.717678 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ca79ba-419a-4177-896a-64acfa225edf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76817cd541294331b0023069092e8e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:39.718282 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ca79ba-419a-4177-896a-64acfa225edf service neutron] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:39.718561 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ca79ba-419a-4177-896a-64acfa225edf service neutron] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:39.718830 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ca79ba-419a-4177-896a-64acfa225edf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:39.728442 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ca79ba-419a-4177-896a-64acfa225edf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:39.729887 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ca79ba-419a-4177-896a-64acfa225edf service neutron] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:39.739163 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1428/2880] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:56:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:56:40.362570 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2832a3e1-8f61-44fc-9a4a-95036a771453 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:40.362884 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2832a3e1-8f61-44fc-9a4a-95036a771453 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:40.363164 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2832a3e1-8f61-44fc-9a4a-95036a771453 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:40.391520 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-2832a3e1-8f61-44fc-9a4a-95036a771453 None None] MFA Rules not processed for user `2277cf311617495cb784285f328451c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:56:40.392730 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-2832a3e1-8f61-44fc-9a4a-95036a771453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:56:40.408685 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:56:40.471073 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1453/2881] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:56:40 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:56:40.483079 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ca79ba-419a-4177-896a-64acfa225edf service neutron] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:40.484684 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-c1ca79ba-419a-4177-896a-64acfa225edf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:56:40.544563 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ca79ba-419a-4177-896a-64acfa225edf service neutron] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:40.548208 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53935aa-5aba-491f-bb7e-4d25424077f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96d512d0c84240918dae6c4a3e8b7b79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:40.548828 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53935aa-5aba-491f-bb7e-4d25424077f7 service cinder] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:40.549105 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53935aa-5aba-491f-bb7e-4d25424077f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:40.549408 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53935aa-5aba-491f-bb7e-4d25424077f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:40.558246 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53935aa-5aba-491f-bb7e-4d25424077f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:40.559669 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53935aa-5aba-491f-bb7e-4d25424077f7 service cinder] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:40.569397 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1429/2882] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:56:40 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:56:41.930958 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a167a182-2394-4dc9-9168-ddcd43637ed6 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:41.931505 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a167a182-2394-4dc9-9168-ddcd43637ed6 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:41.931505 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a167a182-2394-4dc9-9168-ddcd43637ed6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:41.954558 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-a167a182-2394-4dc9-9168-ddcd43637ed6 None None] MFA Rules not processed for user `2277cf311617495cb784285f328451c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:56:41.955778 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-a167a182-2394-4dc9-9168-ddcd43637ed6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:56:41.963488 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1454/2883] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:56:41 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:56:42.212972 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53935aa-5aba-491f-bb7e-4d25424077f7 service cinder] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:42.222329 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53935aa-5aba-491f-bb7e-4d25424077f7 service cinder] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:42.226291 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cdf616-6d22-46c6-bb1d-e70dad0d8ecd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76817cd541294331b0023069092e8e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:42.226926 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cdf616-6d22-46c6-bb1d-e70dad0d8ecd service neutron] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:42.227208 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cdf616-6d22-46c6-bb1d-e70dad0d8ecd service neutron] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:42.227489 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cdf616-6d22-46c6-bb1d-e70dad0d8ecd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:42.229297 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-e9cdf616-6d22-46c6-bb1d-e70dad0d8ecd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:56:42.239140 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cdf616-6d22-46c6-bb1d-e70dad0d8ecd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:42.240610 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cdf616-6d22-46c6-bb1d-e70dad0d8ecd service neutron] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:42.248644 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1430/2884] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:56:42 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:56:42.574368 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a167a182-2394-4dc9-9168-ddcd43637ed6 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:42.576165 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-a167a182-2394-4dc9-9168-ddcd43637ed6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:56:42.651457 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a167a182-2394-4dc9-9168-ddcd43637ed6 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:42.656730 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24251b2b-a3b8-4b40-bca6-80b26ef28f4f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4bd290018394b0d955cd376ce35b77f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:42.657411 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24251b2b-a3b8-4b40-bca6-80b26ef28f4f service placement] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:42.657714 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24251b2b-a3b8-4b40-bca6-80b26ef28f4f service placement] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:42.657988 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24251b2b-a3b8-4b40-bca6-80b26ef28f4f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:42.659817 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-24251b2b-a3b8-4b40-bca6-80b26ef28f4f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:56:42.671648 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24251b2b-a3b8-4b40-bca6-80b26ef28f4f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:42.673231 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24251b2b-a3b8-4b40-bca6-80b26ef28f4f service placement] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:42.683955 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1455/2885] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:56:42 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:56:50.481115 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cdf616-6d22-46c6-bb1d-e70dad0d8ecd service neutron] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:56:50.482590 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-e9cdf616-6d22-46c6-bb1d-e70dad0d8ecd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:56:50.539174 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:56:50.561138 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cdf616-6d22-46c6-bb1d-e70dad0d8ecd service neutron] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:56:50.566638 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5621c5eb-aea8-405e-803b-b0bf3160eca2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd10b49b29df94404a71f43273839c252', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:56:50.567255 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5621c5eb-aea8-405e-803b-b0bf3160eca2 service nova] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:56:50.567535 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5621c5eb-aea8-405e-803b-b0bf3160eca2 service nova] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:56:50.567810 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5621c5eb-aea8-405e-803b-b0bf3160eca2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:56:50.569744 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-5621c5eb-aea8-405e-803b-b0bf3160eca2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:56:50.581646 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5621c5eb-aea8-405e-803b-b0bf3160eca2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:56:50.583208 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5621c5eb-aea8-405e-803b-b0bf3160eca2 service nova] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:56:50.594041 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1431/2886] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:56:50 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:56:52.679886 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=55 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:57:00.589544 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=9 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:57:26.686917 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaaccc3-b3a6-4583-8b93-54ab75889d79 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:57:26.687440 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaaccc3-b3a6-4583-8b93-54ab75889d79 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:57:26.687542 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaaccc3-b3a6-4583-8b93-54ab75889d79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:57:26.706603 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-9aaaccc3-b3a6-4583-8b93-54ab75889d79 None None] MFA Rules not processed for user `c4618ca07ce2408ba90e9eaab0b940c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:57:26.707807 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-9aaaccc3-b3a6-4583-8b93-54ab75889d79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:57:26.763329 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1456/2887] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:57:26 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:57:27.087663 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5621c5eb-aea8-405e-803b-b0bf3160eca2 service nova] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:57:27.089084 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-5621c5eb-aea8-405e-803b-b0bf3160eca2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:57:27.100292 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5621c5eb-aea8-405e-803b-b0bf3160eca2 service nova] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:57:27.104525 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc30278-4fcf-47b5-9360-050920ab01dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76817cd541294331b0023069092e8e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:57:27.105120 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc30278-4fcf-47b5-9360-050920ab01dd service neutron] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:57:27.105423 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc30278-4fcf-47b5-9360-050920ab01dd service neutron] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:57:27.105693 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc30278-4fcf-47b5-9360-050920ab01dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:57:27.107344 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-cbc30278-4fcf-47b5-9360-050920ab01dd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:57:27.117708 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc30278-4fcf-47b5-9360-050920ab01dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:57:27.119169 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc30278-4fcf-47b5-9360-050920ab01dd service neutron] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:57:27.127291 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1432/2888] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:57:27 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:57:36.755776 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=25 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:57:37.123640 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=6 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:57:38.805236 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaaccc3-b3a6-4583-8b93-54ab75889d79 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:57:38.806835 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-9aaaccc3-b3a6-4583-8b93-54ab75889d79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:57:38.823821 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaaccc3-b3a6-4583-8b93-54ab75889d79 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:57:38.829030 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b8b52f-9f24-4a02-bae6-997ac00d431f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4bd290018394b0d955cd376ce35b77f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:57:38.829843 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b8b52f-9f24-4a02-bae6-997ac00d431f service placement] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:57:38.830552 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b8b52f-9f24-4a02-bae6-997ac00d431f service placement] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:57:38.830552 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b8b52f-9f24-4a02-bae6-997ac00d431f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:57:38.831993 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-00b8b52f-9f24-4a02-bae6-997ac00d431f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:57:38.844736 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b8b52f-9f24-4a02-bae6-997ac00d431f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:57:38.846361 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b8b52f-9f24-4a02-bae6-997ac00d431f service placement] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:57:38.857344 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1457/2889] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:57:38 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:57:48.432585 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0ff156-5c41-4c87-9e59-85c011895196 None None] REQUEST_METHOD: `POST` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:57:48.433115 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0ff156-5c41-4c87-9e59-85c011895196 None None] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:57:48.433115 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0ff156-5c41-4c87-9e59-85c011895196 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:57:48.452123 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.auth.core [None req-3a0ff156-5c41-4c87-9e59-85c011895196 None None] MFA Rules not processed for user `2277cf311617495cb784285f328451c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80325) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:57:48.453291 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-3a0ff156-5c41-4c87-9e59-85c011895196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:57:48.460686 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1433/2890] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:57:48 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:57:48.853066 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=6 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:57:50.862146 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b8b52f-9f24-4a02-bae6-997ac00d431f service placement] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:57:50.875954 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b8b52f-9f24-4a02-bae6-997ac00d431f service placement] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:57:50.883111 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee69f268-8343-433b-88f3-9c83595e3ddd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd10b49b29df94404a71f43273839c252', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:57:50.883770 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee69f268-8343-433b-88f3-9c83595e3ddd service nova] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:57:50.884053 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee69f268-8343-433b-88f3-9c83595e3ddd service nova] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:57:50.884327 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee69f268-8343-433b-88f3-9c83595e3ddd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:57:50.886191 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-ee69f268-8343-433b-88f3-9c83595e3ddd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:57:50.899345 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee69f268-8343-433b-88f3-9c83595e3ddd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:57:50.901109 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee69f268-8343-433b-88f3-9c83595e3ddd service nova] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:57:50.913613 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1458/2891] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:57:50 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:57:58.446687 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=3 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:58:00.908847 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=6 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:58:47.238897 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0ff156-5c41-4c87-9e59-85c011895196 None None] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:58:47.252019 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0ff156-5c41-4c87-9e59-85c011895196 None None] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:58:47.257085 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e59a87-2ee1-4e63-bbbd-168542cbb484 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4bd290018394b0d955cd376ce35b77f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:58:47.257742 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e59a87-2ee1-4e63-bbbd-168542cbb484 service placement] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:58:47.258018 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e59a87-2ee1-4e63-bbbd-168542cbb484 service placement] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:58:47.258294 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e59a87-2ee1-4e63-bbbd-168542cbb484 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:58:47.260165 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-28e59a87-2ee1-4e63-bbbd-168542cbb484 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:58:47.272813 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e59a87-2ee1-4e63-bbbd-168542cbb484 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:58:47.274965 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e59a87-2ee1-4e63-bbbd-168542cbb484 service placement] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:58:47.285970 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1434/2892] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:58:47 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:58:47.291489 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee69f268-8343-433b-88f3-9c83595e3ddd service nova] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:58:47.302158 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee69f268-8343-433b-88f3-9c83595e3ddd service nova] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:58:47.307938 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5574db65-1cbe-4076-a8a1-7c47859ac2ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:58:47.308604 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5574db65-1cbe-4076-a8a1-7c47859ac2ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:58:47.308889 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5574db65-1cbe-4076-a8a1-7c47859ac2ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:58:47.309395 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5574db65-1cbe-4076-a8a1-7c47859ac2ee admin admin] PATH_INFO: `/v3/users/2277cf311617495cb784285f328451c5` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:58:47.310778 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5574db65-1cbe-4076-a8a1-7c47859ac2ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=2277cf311617495cb784285f328451c5)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:58:47.312683 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5574db65-1cbe-4076-a8a1-7c47859ac2ee admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:58:47.360592 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-5574db65-1cbe-4076-a8a1-7c47859ac2ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2277cf311617495cb784285f328451c5'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:58:47.367777 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-5574db65-1cbe-4076-a8a1-7c47859ac2ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2277cf311617495cb784285f328451c5'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:58:47.368450 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-5574db65-1cbe-4076-a8a1-7c47859ac2ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2277cf311617495cb784285f328451c5'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:58:47.375508 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-5574db65-1cbe-4076-a8a1-7c47859ac2ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2277cf311617495cb784285f328451c5'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:58:47.387836 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-5574db65-1cbe-4076-a8a1-7c47859ac2ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2277cf311617495cb784285f328451c5'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:58:47.393969 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1459/2893] 10.222.0.89 () {58 vars in 1224 bytes} [Thu Jul 28 17:58:47 2022] DELETE /identity/v3/users/2277cf311617495cb784285f328451c5 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:58:47.572871 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e59a87-2ee1-4e63-bbbd-168542cbb484 service placement] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:58:47.584059 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e59a87-2ee1-4e63-bbbd-168542cbb484 service placement] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:58:47.641967 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f896f8ce-e3c6-41b0-9ed6-3c569ad61f19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:58:47.642529 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f896f8ce-e3c6-41b0-9ed6-3c569ad61f19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:58:47.642808 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f896f8ce-e3c6-41b0-9ed6-3c569ad61f19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:58:47.643081 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f896f8ce-e3c6-41b0-9ed6-3c569ad61f19 admin admin] PATH_INFO: `/v3/projects/6ed3b5ffc1dd4fb9a814a967d1c9e8dc` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:58:47.644229 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f896f8ce-e3c6-41b0-9ed6-3c569ad61f19 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ed3b5ffc1dd4fb9a814a967d1c9e8dc)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:58:47.646185 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f896f8ce-e3c6-41b0-9ed6-3c569ad61f19 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:58:47.692887 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-f896f8ce-e3c6-41b0-9ed6-3c569ad61f19 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ed3b5ffc1dd4fb9a814a967d1c9e8dc', 'request_id': 'req-f896f8ce-e3c6-41b0-9ed6-3c569ad61f19'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:58:47.699111 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-f896f8ce-e3c6-41b0-9ed6-3c569ad61f19 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ed3b5ffc1dd4fb9a814a967d1c9e8dc', 'request_id': 'req-f896f8ce-e3c6-41b0-9ed6-3c569ad61f19'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:58:47.704598 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-f896f8ce-e3c6-41b0-9ed6-3c569ad61f19 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ed3b5ffc1dd4fb9a814a967d1c9e8dc', 'request_id': 'req-f896f8ce-e3c6-41b0-9ed6-3c569ad61f19'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:58:47.717467 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-f896f8ce-e3c6-41b0-9ed6-3c569ad61f19 admin admin] The token cache is being invalidate because project 6ed3b5ffc1dd4fb9a814a967d1c9e8dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80325) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 17:58:47.717765 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-f896f8ce-e3c6-41b0-9ed6-3c569ad61f19 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:58:47.718390 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-f896f8ce-e3c6-41b0-9ed6-3c569ad61f19 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 17:58:47.719900 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1435/2894] 10.222.0.89 () {58 vars in 1233 bytes} [Thu Jul 28 17:58:47 2022] DELETE /identity/v3/projects/6ed3b5ffc1dd4fb9a814a967d1c9e8dc => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 17:58:57.390165 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=29,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:58:57.714381 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=57,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:59:37.765009 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981f2f25-fea8-4591-b1a9-248102eddc14 None None] REQUEST_METHOD: `POST` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:59:37.765620 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981f2f25-fea8-4591-b1a9-248102eddc14 None None] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:59:37.765620 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981f2f25-fea8-4591-b1a9-248102eddc14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:59:37.784577 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.auth.core [None req-981f2f25-fea8-4591-b1a9-248102eddc14 None None] MFA Rules not processed for user `c4618ca07ce2408ba90e9eaab0b940c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80324) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 17:59:37.785910 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-981f2f25-fea8-4591-b1a9-248102eddc14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:59:37.842286 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1460/2895] 10.222.0.89 () {58 vars in 958 bytes} [Thu Jul 28 17:59:37 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 17:59:41.998845 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f896f8ce-e3c6-41b0-9ed6-3c569ad61f19 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:59:42.000311 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-f896f8ce-e3c6-41b0-9ed6-3c569ad61f19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:59:42.078568 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f896f8ce-e3c6-41b0-9ed6-3c569ad61f19 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:59:42.084339 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66446ce7-8f1e-4e4d-b5a3-cce0ba030394 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd10b49b29df94404a71f43273839c252', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:59:42.084954 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66446ce7-8f1e-4e4d-b5a3-cce0ba030394 service nova] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:59:42.085265 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66446ce7-8f1e-4e4d-b5a3-cce0ba030394 service nova] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:59:42.085550 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66446ce7-8f1e-4e4d-b5a3-cce0ba030394 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:59:42.088004 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-66446ce7-8f1e-4e4d-b5a3-cce0ba030394 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:59:42.098563 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66446ce7-8f1e-4e4d-b5a3-cce0ba030394 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:59:42.100191 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66446ce7-8f1e-4e4d-b5a3-cce0ba030394 service nova] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:59:42.109560 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1436/2896] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:59:41 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:59:43.338856 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981f2f25-fea8-4591-b1a9-248102eddc14 None None] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:59:43.340252 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-981f2f25-fea8-4591-b1a9-248102eddc14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:59:43.396218 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981f2f25-fea8-4591-b1a9-248102eddc14 None None] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:59:43.401038 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b767af-bdde-4f34-913d-77372c4c15d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76817cd541294331b0023069092e8e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:59:43.401684 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b767af-bdde-4f34-913d-77372c4c15d6 service neutron] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:59:43.401965 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b767af-bdde-4f34-913d-77372c4c15d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:59:43.402241 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b767af-bdde-4f34-913d-77372c4c15d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:59:43.404031 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-55b767af-bdde-4f34-913d-77372c4c15d6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:59:43.419270 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b767af-bdde-4f34-913d-77372c4c15d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:59:43.420691 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b767af-bdde-4f34-913d-77372c4c15d6 service neutron] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:59:43.432490 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1461/2897] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:59:43 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:59:49.304797 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66446ce7-8f1e-4e4d-b5a3-cce0ba030394 service nova] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 17:59:49.306234 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-66446ce7-8f1e-4e4d-b5a3-cce0ba030394 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:59:49.318230 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66446ce7-8f1e-4e4d-b5a3-cce0ba030394 service nova] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 17:59:49.322344 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faf4e58-0082-4c47-85b3-3f9b89e76ee2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76817cd541294331b0023069092e8e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 17:59:49.322950 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faf4e58-0082-4c47-85b3-3f9b89e76ee2 service neutron] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 17:59:49.323231 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faf4e58-0082-4c47-85b3-3f9b89e76ee2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 17:59:49.323553 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faf4e58-0082-4c47-85b3-3f9b89e76ee2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 17:59:49.325359 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-3faf4e58-0082-4c47-85b3-3f9b89e76ee2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 17:59:49.339412 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3faf4e58-0082-4c47-85b3-3f9b89e76ee2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 17:59:49.340759 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3faf4e58-0082-4c47-85b3-3f9b89e76ee2 service neutron] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 17:59:49.352361 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1437/2898] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 17:59:49 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 17:59:53.426999 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=51 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 17:59:59.346798 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=40 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:00:41.184076 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b767af-bdde-4f34-913d-77372c4c15d6 service neutron] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:00:41.193065 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b767af-bdde-4f34-913d-77372c4c15d6 service neutron] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:00:41.197520 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ea6f77-faff-4f97-ba8b-98be8562d1a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76817cd541294331b0023069092e8e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:00:41.198154 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ea6f77-faff-4f97-ba8b-98be8562d1a4 service neutron] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:00:41.198442 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ea6f77-faff-4f97-ba8b-98be8562d1a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:00:41.198716 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ea6f77-faff-4f97-ba8b-98be8562d1a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:00:41.200488 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-80ea6f77-faff-4f97-ba8b-98be8562d1a4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:00:41.211538 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ea6f77-faff-4f97-ba8b-98be8562d1a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:00:41.212969 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ea6f77-faff-4f97-ba8b-98be8562d1a4 service neutron] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:00:41.222440 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1462/2899] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 18:00:41 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:00:51.217640 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=6 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:01:28.696660 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faf4e58-0082-4c47-85b3-3f9b89e76ee2 service neutron] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:01:28.699175 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-3faf4e58-0082-4c47-85b3-3f9b89e76ee2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:01:28.720354 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faf4e58-0082-4c47-85b3-3f9b89e76ee2 service neutron] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:01:28.726803 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c928bafe-c987-448d-8fa1-018714b414d9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd10b49b29df94404a71f43273839c252', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:01:28.727429 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c928bafe-c987-448d-8fa1-018714b414d9 service nova] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:01:28.727708 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c928bafe-c987-448d-8fa1-018714b414d9 service nova] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:01:28.727981 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c928bafe-c987-448d-8fa1-018714b414d9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:01:28.729789 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-c928bafe-c987-448d-8fa1-018714b414d9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:01:28.743806 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c928bafe-c987-448d-8fa1-018714b414d9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:01:28.745425 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c928bafe-c987-448d-8fa1-018714b414d9 service nova] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:01:28.758970 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1438/2900] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 18:01:28 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:01:38.752655 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=6 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:01:40.961570 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ea6f77-faff-4f97-ba8b-98be8562d1a4 service neutron] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:01:40.970338 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ea6f77-faff-4f97-ba8b-98be8562d1a4 service neutron] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:01:40.974717 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7dae42-78da-4bf6-a5cd-e90dd5d6a114 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76817cd541294331b0023069092e8e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:01:40.975337 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7dae42-78da-4bf6-a5cd-e90dd5d6a114 service neutron] REQUEST_METHOD: `GET` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:01:40.975625 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7dae42-78da-4bf6-a5cd-e90dd5d6a114 service neutron] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:01:40.975906 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7dae42-78da-4bf6-a5cd-e90dd5d6a114 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:01:40.977745 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-da7dae42-78da-4bf6-a5cd-e90dd5d6a114 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:01:41.041351 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7dae42-78da-4bf6-a5cd-e90dd5d6a114 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:01:41.042770 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7dae42-78da-4bf6-a5cd-e90dd5d6a114 service neutron] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:01:41.052660 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1463/2901] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 18:01:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 18:01:47.238667 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c928bafe-c987-448d-8fa1-018714b414d9 service nova] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:01:47.241011 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-c928bafe-c987-448d-8fa1-018714b414d9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:01:47.256594 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c928bafe-c987-448d-8fa1-018714b414d9 service nova] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:01:47.262098 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2233f161-ba1c-4c76-bafe-2847d30f1797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:01:47.262728 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2233f161-ba1c-4c76-bafe-2847d30f1797 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:01:47.263003 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2233f161-ba1c-4c76-bafe-2847d30f1797 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:01:47.263273 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2233f161-ba1c-4c76-bafe-2847d30f1797 admin admin] PATH_INFO: `/v3/users/c4618ca07ce2408ba90e9eaab0b940c9` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:01:47.264433 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2233f161-ba1c-4c76-bafe-2847d30f1797 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c4618ca07ce2408ba90e9eaab0b940c9)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:01:47.266373 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2233f161-ba1c-4c76-bafe-2847d30f1797 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:01:47.316229 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-2233f161-ba1c-4c76-bafe-2847d30f1797 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c4618ca07ce2408ba90e9eaab0b940c9'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:01:47.323641 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-2233f161-ba1c-4c76-bafe-2847d30f1797 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c4618ca07ce2408ba90e9eaab0b940c9'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:01:47.331915 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-2233f161-ba1c-4c76-bafe-2847d30f1797 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c4618ca07ce2408ba90e9eaab0b940c9'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:01:47.332741 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-2233f161-ba1c-4c76-bafe-2847d30f1797 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c4618ca07ce2408ba90e9eaab0b940c9'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:01:47.333974 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-2233f161-ba1c-4c76-bafe-2847d30f1797 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c4618ca07ce2408ba90e9eaab0b940c9'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:01:47.352417 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1439/2902] 10.222.0.89 () {58 vars in 1224 bytes} [Thu Jul 28 18:01:47 2022] DELETE /identity/v3/users/c4618ca07ce2408ba90e9eaab0b940c9 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:01:47.534014 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7dae42-78da-4bf6-a5cd-e90dd5d6a114 service neutron] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:01:47.545281 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7dae42-78da-4bf6-a5cd-e90dd5d6a114 service neutron] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:01:47.603711 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb160ce-26e2-4527-a465-40d3b30c9a31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:01:47.604362 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb160ce-26e2-4527-a465-40d3b30c9a31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:01:47.604631 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb160ce-26e2-4527-a465-40d3b30c9a31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:01:47.604914 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb160ce-26e2-4527-a465-40d3b30c9a31 admin admin] PATH_INFO: `/v3/projects/1f721aa2c35842beba2060fde666b2a1` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:01:47.606098 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb160ce-26e2-4527-a465-40d3b30c9a31 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f721aa2c35842beba2060fde666b2a1)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:01:47.608004 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb160ce-26e2-4527-a465-40d3b30c9a31 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:01:47.654144 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-4bb160ce-26e2-4527-a465-40d3b30c9a31 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f721aa2c35842beba2060fde666b2a1', 'request_id': 'req-4bb160ce-26e2-4527-a465-40d3b30c9a31'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:01:47.659433 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-4bb160ce-26e2-4527-a465-40d3b30c9a31 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f721aa2c35842beba2060fde666b2a1', 'request_id': 'req-4bb160ce-26e2-4527-a465-40d3b30c9a31'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:01:47.666176 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-4bb160ce-26e2-4527-a465-40d3b30c9a31 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f721aa2c35842beba2060fde666b2a1', 'request_id': 'req-4bb160ce-26e2-4527-a465-40d3b30c9a31'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:01:47.680228 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-4bb160ce-26e2-4527-a465-40d3b30c9a31 admin admin] The token cache is being invalidate because project 1f721aa2c35842beba2060fde666b2a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80324) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:01:47.680518 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-4bb160ce-26e2-4527-a465-40d3b30c9a31 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:01:47.681139 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-4bb160ce-26e2-4527-a465-40d3b30c9a31 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:01:47.682668 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1464/2903] 10.222.0.89 () {58 vars in 1233 bytes} [Thu Jul 28 18:01:47 2022] DELETE /identity/v3/projects/1f721aa2c35842beba2060fde666b2a1 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:01:47.686941 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2233f161-ba1c-4c76-bafe-2847d30f1797 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:01:47.688353 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-2233f161-ba1c-4c76-bafe-2847d30f1797 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:01:47.701455 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2233f161-ba1c-4c76-bafe-2847d30f1797 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:01:47.759348 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38639ba7-a5b6-4355-b175-019b2a208b53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:01:47.759991 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38639ba7-a5b6-4355-b175-019b2a208b53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:01:47.760273 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38639ba7-a5b6-4355-b175-019b2a208b53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:01:47.760550 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38639ba7-a5b6-4355-b175-019b2a208b53 admin admin] PATH_INFO: `/v3/users/07d6b2e80ab1411594bacded08511ed4` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:01:47.761757 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38639ba7-a5b6-4355-b175-019b2a208b53 admin admin] RBAC: Authorizing `identity:delete_user(user_id=07d6b2e80ab1411594bacded08511ed4)` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:01:47.763676 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38639ba7-a5b6-4355-b175-019b2a208b53 admin admin] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:01:47.808784 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-38639ba7-a5b6-4355-b175-019b2a208b53 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07d6b2e80ab1411594bacded08511ed4'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:01:47.815765 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-38639ba7-a5b6-4355-b175-019b2a208b53 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07d6b2e80ab1411594bacded08511ed4'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:01:47.823857 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-38639ba7-a5b6-4355-b175-019b2a208b53 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07d6b2e80ab1411594bacded08511ed4'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:01:47.824686 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-38639ba7-a5b6-4355-b175-019b2a208b53 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07d6b2e80ab1411594bacded08511ed4'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:01:47.825342 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.notifications [None req-38639ba7-a5b6-4355-b175-019b2a208b53 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07d6b2e80ab1411594bacded08511ed4'} {{(pid=80325) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:01:47.835926 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:01:47.847850 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1440/2904] 10.222.0.89 () {58 vars in 1224 bytes} [Thu Jul 28 18:01:47 2022] DELETE /identity/v3/users/07d6b2e80ab1411594bacded08511ed4 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:01:48.023588 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb160ce-26e2-4527-a465-40d3b30c9a31 admin admin] Authenticating user token {{(pid=80324) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:01:48.025327 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.fernet_utils [None req-4bb160ce-26e2-4527-a465-40d3b30c9a31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80324) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:01:48.040695 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb160ce-26e2-4527-a465-40d3b30c9a31 admin admin] Validating token access rules against request {{(pid=80324) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:01:48.062434 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:01:48.100753 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17f1c41-0482-42cc-9593-c34870754735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbc7ab320041413396df1eef4db728ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd68024df9ad444518749a9b417c3b67b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80324) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:01:48.101419 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17f1c41-0482-42cc-9593-c34870754735 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:01:48.101699 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17f1c41-0482-42cc-9593-c34870754735 admin admin] SCRIPT_NAME: `/identity` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:01:48.101967 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17f1c41-0482-42cc-9593-c34870754735 admin admin] PATH_INFO: `/v3/projects/a36fe479443540dfbecf9373a72ca18d` {{(pid=80324) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:01:48.103118 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17f1c41-0482-42cc-9593-c34870754735 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a36fe479443540dfbecf9373a72ca18d)` {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:01:48.105002 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17f1c41-0482-42cc-9593-c34870754735 admin admin] RBAC: Authorization granted {{(pid=80324) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:01:48.152838 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-c17f1c41-0482-42cc-9593-c34870754735 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a36fe479443540dfbecf9373a72ca18d', 'request_id': 'req-c17f1c41-0482-42cc-9593-c34870754735'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:01:48.158594 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-c17f1c41-0482-42cc-9593-c34870754735 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a36fe479443540dfbecf9373a72ca18d', 'request_id': 'req-c17f1c41-0482-42cc-9593-c34870754735'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:01:48.165420 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-c17f1c41-0482-42cc-9593-c34870754735 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a36fe479443540dfbecf9373a72ca18d', 'request_id': 'req-c17f1c41-0482-42cc-9593-c34870754735'} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:01:48.180010 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-c17f1c41-0482-42cc-9593-c34870754735 admin admin] The token cache is being invalidate because project a36fe479443540dfbecf9373a72ca18d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80324) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 18:01:48.180297 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-c17f1c41-0482-42cc-9593-c34870754735 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:01:48.180915 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG keystone.notifications [None req-c17f1c41-0482-42cc-9593-c34870754735 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80324) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 18:01:48.182459 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: [pid: 80324|app: 0|req: 1465/2905] 10.222.0.89 () {58 vars in 1233 bytes} [Thu Jul 28 18:01:48 2022] DELETE /identity/v3/projects/a36fe479443540dfbecf9373a72ca18d => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 18:01:57.848269 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=3,keystone:DELETE=1 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:01:58.176266 dv-cinder-813229-13-iscsi devstack@keystone.service[80324]: DEBUG dbcounter [-] [80324] Writing DB stats keystone:SELECT=40,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80324) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 18:02:48.488749 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38639ba7-a5b6-4355-b175-019b2a208b53 admin admin] Authenticating user token {{(pid=80325) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 18:02:48.490215 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-38639ba7-a5b6-4355-b175-019b2a208b53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:48.630478 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38639ba7-a5b6-4355-b175-019b2a208b53 admin admin] Validating token access rules against request {{(pid=80325) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 18:02:48.690839 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a439e69-6f59-40ac-a05a-7ba2c7d4ebc6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4bd290018394b0d955cd376ce35b77f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04434e8535a64044996316f1d17dc0bc', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80325) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 18:02:48.691196 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a439e69-6f59-40ac-a05a-7ba2c7d4ebc6 service placement] REQUEST_METHOD: `GET` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 18:02:48.691486 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a439e69-6f59-40ac-a05a-7ba2c7d4ebc6 service placement] SCRIPT_NAME: `/identity` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 18:02:48.691766 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a439e69-6f59-40ac-a05a-7ba2c7d4ebc6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80325) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 18:02:48.693637 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.fernet_utils [None req-6a439e69-6f59-40ac-a05a-7ba2c7d4ebc6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80325) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 18:02:48.705304 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a439e69-6f59-40ac-a05a-7ba2c7d4ebc6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 18:02:48.706876 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a439e69-6f59-40ac-a05a-7ba2c7d4ebc6 service placement] RBAC: Authorization granted {{(pid=80325) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 18:02:48.716777 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: [pid: 80325|app: 0|req: 1441/2906] 10.222.0.89 () {60 vars in 1365 bytes} [Thu Jul 28 18:02:48 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 18:02:58.712509 dv-cinder-813229-13-iscsi devstack@keystone.service[80325]: DEBUG dbcounter [-] [80325] Writing DB stats keystone:SELECT=89 {{(pid=80325) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}